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The  National  Bureau  of  Standards1  was  established  by  an  act  of  Congress  March  3,  1901. 
The  Bureau's  overall  goal  is  to  strengthen  and  advance  the  Nation's  science  and  technology 
and  facilitate  their  effective  application  for  public  benefit.  To  this  end,  the  Bureau  conducts 
research  and  provides:  (1)  a  basis  for  the  Nation's  physical  measurement  system,  (2)  scientific 
and  technological  services  for  industry  and  government,  (3)  a  technical  basis  for  equity  in  trade, 
and  (4)  technical  services  to  promote  public  safety.  The  Bureau  consists  of  the  Institute  for 
Basic  Standards,  the  Institute  for  Materials  Research,  the  Institute  for  Applied  Technology, 
the  Institute  for  Computer  Sciences  and  Technology,  and  the  Office  for  Information  Programs. 

THE  INSTITUTE  FOR  BASIC  STANDARDS  provides  the  central  basis  within  the  United 
States  of  a  complete  and  consistent  system  of  physical  measurement;  coordinates  that  system 
with  measurement  systems  of  other  nations;  and  furnishes  essential  services  leading  to  accurate 
and  uniform  physical  measurements  throughout  the  Nation's  scientific  community,  industry, 
and  commerce.  The  Institute  ^consists  of  a  Center  for  Radiation  Research,  an  Office  of  Meas- 
urement Services  and  the  following  divisions: 
Applied  Mathematics  —  Electricity  —  Mechanics  —  Heat  —  Optical  Physics  —  Nuclear 
Sciences "  —  Applied  Radiation  '-'  —  Quantum  Electronics 3  —  Electromagnetics 3  —  Time 
and  Frequency  *  —  Laboratory  Astrophysics  "  ■ —  Cryogenics ". 

THE  INSTITUTE  FOR  MATERIALS  RESEARCH  conducts  materials  research  leading  to 
improved  methods  of  measurement,  standards,  and  data  on  the  properties  of  well-characterized 
materials  needed  by  industry,  commerce,  educational  institutions,  and  Government;  provides 
advisory  and  research  services  to  other  Government  agencies;  and  develops,  produces,  and 
distributes  standard  reference  materials.  The  Institute  consists  of  the  Office  of  Standard 
Reference  Materials  and  the  following  divisions: 

Analytical   Chemistry  —  Polymers  —  Metallurgy  —  Inorganic  Materials  —  Reactor 

Radiation  —  Physical  Chemistry. 

THE  INSTITUTE  FOR  APPLIED  TECHNOLOGY  provides  technical  services  to  promote 
the  use  of  available  technology  and  to  facilitate  technological  innovation  in  industry  and 
Government;  cooperates  with  public  and  private  organizations  leading  to  the  development  of 
technological  standards  (including  mandatory  safety  standards),  codes  and  methods  of  test; 
and  provides  technical  advice  and  services  to  Government  agencies  upon  request.  The  Institute 
consists  of  a  Center  for  Building  Technology  and  the  following  divisions  and  offices: 

Engineering  and  Product  Standards  —  Weights  and  Measures  —  Invention  and  Innova- 
tion —  Product  Evaluation  Technology  —  Electronic  Technology  —  Technical  Analysis 
—  Measurement  Engineering  —  Structures,  Materials,  and  Life  Safety4  —  Building 
Environment  *  —  Technical  Evaluation  and  Application  *  —  Fire  Technology. 

THE  INSTITUTE  FOR  COMPUTER  SCIENCES  AND  TECHNOLOGY  conducts  research 
and  provides  technical  services  designed  to  aid  Government  agencies  in  improving  cost  effec- 
tiveness in  the  conduct  of  their  programs  through  the  selection,  acquisition,  and  effective 
utilization  of  automatic  data  processing  equipment;  and  serves  as  the  principal  focus  within 
the  executive  branch  for  the  development  of  Federal  standards  for  automatic  data  processing 
equipment,  techniques,  and  computer  languages.  The  Institute  consists  of  the  following 
divisions: 

Computer  Services  —  Systems  and  Software  —  Computer  Systems  Engineering  —  Informa- 
tion Technology. 

THE  OFFICE  FOR  INFORMATION  PROGRAMS  promotes  optimum  dissemination  and 
accessibility  of  scientific  information  generated  within  NBS  and  other  agencies  of  the  Federal 
Government;  promotes  the  development  of  the  National  Standard  Reference  Data  System  and 
a  system  of  information  analysis  centers  dealing  with  the  broader  aspects  of  the*  National 
Measurement  System;  provides  appropriate  services  to  ensure  that  the  NBS  staff  has  optimum 
accessibility  to  the  scientific  information  of  the  world.  The  Office  consists  of  the  following 
organizational  units: 

Office  of  Standard  Reference  Data  —  Office  of  Information  Activities  —  Office  of  Technical 

Publications  —  Library  —  Office  of  International  Relations. 
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3  Part  of  the  Center  for  Radiation  Research.  y 
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Abstract 

Recognizing  the  proliferation  of  sophisticated  modern  electronic  instrumentation  in  the  field  of 
electrical  measurements,  the  Electricity  Division  of  the  National  Bureau  of  Standards  recently  initi- 
ated a  new  program  in  the  general  area  of  dynamic  measurements  and  standards  in  support  of  such 
instrumentation.    Recognizing  further  that  the  vastness  and  complexity  of  the  field  would  require,  at 
the  earliest  stages  of  the  program,  identification  of  the  most  critical  problem  areas,  the  Electricity 
Division  held  a  workshop  on  23  and  24  September,   1974,  at  the  Bureau's  Gaithersburg  site,  to  assist 
it  in  ascertaining  just  what  these  areas  in  fact  were.    The  basic  idea  of  the  Workshop  was  to  bring 
together  a  broadly  representative  group  of  some  twenty-five  leading  manufacturers  and  prime  users, 
working  in  a  free  and  open  atmosphere,  in  order  to  have  them  delineate  the  present  and  future  criti- 
cal support  needs  in  the  field  of  dynamic  electrical  measurements  for  modern  electronic  instrumen- 
tation, with  emphasis  on  physical  standards,  standardized  measurement  methods,  new  calibration 
and  measurement  assurance  services,  relevant  data,  and  most  important,  new  measurement  meth- 
odologies.   The  overall  objectives  of  the  Workshop  were  generally  met,  and  a  number  of  significant 
specific;  programs  and  projects  consistent  with  the  mission  of  the  Electricity  Division  were  identi- 
fied.   Three  categories  broadly  cover  the  needs  as  defined  at  the  Workshop: 

1.  The  need  to  introduce  time  as  a  measurement  parameter  has  resulted  from  the  require- 
ments of  automatic  test  and  control  systems.    Specific  areas  considered  critical  are: 

a.  Pulsed  component  measurements. 

b.  Dynamic  performance  characterization  for  modern  signal  conditioning  and  data  con- 
version devices:   Digital-to-analog  and  analog-to-digital  converters,  sample-and- 
hold  amplifiers,  comparators,  etc.    Required  measurements  include  settling,  aper- 
ture and  acquisition  times.    Basic  new  capabilities  will  be  needed,  including  preci- 
sion, non- sinusoidal  waveform  generation  and  "standard"  digital-to-analog  convert- 
ers. 

c.  Methodologies  and  techniques  for  characterizing  precision  AC  and  DC  sources  and 
measurement  devices  with  respect  to  settling  time. 

2.  The  emergence  of  measurements  into  the  "real -world"  or  the  production  line  via  the 
automatic  system  has  introduced  a  host  of  new  parameters.    Areas  requiring  significant 
effort  include: 

a.  Investigation  of  transportable  standards  for  validation  of  static  and  dynamic  system 
performance:   AC  and  DC  voltage,  impedance,  pulses,  settling  times. 

b.  Techniques  for  characterizing  sources  and  measurement  devices  with  respect  to 
switched  or  dynamic  loading. 

c.  Prediction  of  long-term  performance,  reliability  and  lifetime  from  short-term 
evaluation  for  a  host  of  passive  components,  semiconductors,  and  signal  sources. 

d.  Noise  standards  and  methodologies. 

e.  Continuing  effort  to  improve  transportable  DC  standards,  and  dissemination  at 
higher  voltage  levels. 

f.  Characterization  of  the  AC  line  voltage  -  waveforms,  under-  and  over-voltages, 
transients,  dynamic  impedance,  etc. 

g.  Inclusion  of  a  capability  for  environmental  control  and  variation  for  devices  and 
instruments  under  calibration,  in  all  new  work  undertaken  by  the  Bureau. 

h.      Extension  of  Measurement  Assurance  Programs  (MAP's),  especially  into  the  above 
new  areas. 

3.  Recent  work  at  the  leading  edge  of  measurement  technology  in  electronics  has  resulted 
in  the  need  for  new,  or  higher  accuracy  measurements.     For  example: 


in 


a.  Increased  dependence  on  capacitors  as  storage  devices  has  made  dielectric  hystere- 
sis a  parameter  of  importance.    Significant  measurement  work  should  be  started  in 
this  area. 

b.  Phase  difference  measurements  to  extremely  high  accuracies  are  needed  to  calibrate 
industrial  instrumentation. 

c.  Higher  accuracy  power  measurements,  particularly  for  non- sinusoidal  waveforms, 
are  needed  to  calibrate  instrumentation  used  in  the  field. 

d.  Non- sinusoidal,  high  crest-factor  precision  signals  are  needed  to  calibrate  true  RMS 
converters  and  meters. 

e.  High  accuracy  electronic  instrumentation  requires  reduction  in  AC  calibration  uncer- 
tainties to  10  ppm,  at  least  over  the  audio  range. 

Longer-term,  "frontier"  areas  in  each  of  the  above  groups  were  also  identified.    They  are  not 
specified  in  the  above  listings  since  they  involve  work  at  still  more  basic  levels,  on  ground  that  is 
presently  terra  incognita.    Nevertheless,  they  furnish  insight  into  the  overall  technical/philosophic 
climate,  and  are  useful  for  that  reason.    Items  in  this  category  include: 

1.  Measurement  problems  in  systems,  including  effects  of  history,  measurement  interac- 
tions, self- calibration,  etc. 

2.  Evaluation  and  standardization  of  system  software. 

3.  Calibration  of  medical  equipment. 

4.  Replacement  of  standard  cells. 

5.  Replacement  of  thermal  AC/DC  converters. 

6.  Non-swept  but  broad -response  measurements  -  impulse,  noise,  other. 

7.  Basic  work  on  the  physics  of  dielectric  absorption;  component  drift;  device  noise;  failure 
mechanisms;  switches  and  relays. 

8.  Provision  of  improved  measurement  accuracy  for  virtually  all  dynamic  measurements,  at 
the  wafer  probe  level. 

In  addition  to  the  identification  of  these  specific  and  general  needs,  the  Workshop  provided 
strong  evidence  that  both  the  manufacturing  and  user  communities  in  the  electronic  instrumentation 
field  look  to  the  Electricity  Division  to  provide  the  technological  leadership  in  these  newer  areas  that 
it  has  previously  furnished  in  the  more  traditional  electrical  measurement  areas.    Indeed,  there  was 
a  good  deal  of  support  for  the  idea  of  keeping  the  electronics  community  informed  of  Division  work- 
in-progress,  with  emphasis  on  projects  targeted  by  the  Workshop. 

Keywords:     Data  conversion;  dynamic  measurements;  electrical  measurements; 

electronic  instrumentation;  signal  conditioning;  systems;  time  domain. 
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Foreword 

The  National  Bureau  of  Standards  (NBS)  is  becoming  increasingly  aware  that  its  measurement 
services,  related  research  and  other  activities  must  be  oriented  towards  the  solution  of  the  mea- 
surement community's  "real-world"  problems  if  NBS  wishes  to  make  a  significant  contribution  to 
meeting  the  country's  needs  in  those  areas  containing  a  large  scientific  and  technological  component, 
for  example,  increased  productivity  and  foreign  trade,  environmental  pollution  control,  improved 
health  care  delivery,  solution  of  the  so  called  "energy  crisis",  etc.    The  Electricity  Division  partic- 
ularly recognizes  the  increased  burden  this  new  awareness  places  upon  it,  especially  in  the  area  of 
measurement  and  standards  support  for  modern  electronic  instrumentation,  since  such  instrumenta- 
tion is  rapidly  becoming  all  pervasive  and  a  key  element  in  the  solution  of  our  society's  most  criti- 
cal technical  problems. 

In  keeping  with  this  point  of  view,  and  the  current  trend  in  electrical  measurements  towards 
the  wide  utilization  of  electronic  instrumentation,  often  in  conjunction  with  digital  computers,  and 
more  often  than  not  high  speed  measurements  in  the  time  domain,  the  Division  has  established  over 
the  last  year  or  so  a  nascent  program  in  the  general  area  of  dynamic  measurements  and  standards 
for  modern  electronic  instrumentation.    Present  activities  include  development  of  a  computer-based 
sampling  watt- system;  new  methods  for  accurate  phase  angle  measurements;  calibration  and  support 
methods  for  DC  calibrators;  characterization  and  calibration  of  solid  state  voltage  sources  and  dis- 
semination of  the  unit  of  voltage  at  levels  higher  than  one  volt;  static  and  dynamic  test  sets  for  char- 
acterizing analog-to -digital  and  digital-to-analog  converters;  and  automation  of  the  Division's  cali- 
bration services  (capacitors,  resistors,  standard  cells).    Each  of  these  projects  will  not  only  result 
in  an  improved  measurement  method  and/or  measurement  service,  but  will  provide  Division  person- 
nel with  valuable,  "hands  on"  experience  in  the  field  of  modern  electronic  instrumentation,  auto- 
mated measurements,  and  automated  test  systems. 

It  is  important  to  realize  that  these  new  activities  are  consistent  with  and  complement  the  Divi- 
sion's overall  responsibility  for  establishing,  maintaining  and  disseminating  the  electrical  units  in 
the  United  States,  for  ensuring  that  electrical  measurements  made  throughout  the  country  are  reli- 
able and  consistent,  especially  with  those  made  in  other  countries,  and  of  advancing  the  state-of-the- 
art,  that  is,  of  ensuring  that  useful  and  usable  electrical  measurement  methods  and  standards  are 
available  when  needed. 

It  was  clear  at  the  time  the  new  program  was  initiated  that  the  work  the  Division  was  undertak- 
ing in  this  area  was  just  a  first  step  into  a  very  extensive  and  complex  technology.    Indeed,  the 
field's  scope  and  multiplicity  of  needs  was  found  to  be  so  vast  that  it  was  decided  at  the  outset  that 
extensive,  broadly-based  and  critically  evaluated  input  from  the  potential  beneficiaries  of  the  Divi- 
sion's work  in  this  area  had  to  be  available  before  the  program  could  be  developed  further.    A  work- 
shop was  thus  planned  to  provide  this  input.    The  basic  idea  was  to  bring  together  a  representative 
group  of  leading  manufacturers  and  prime  users,  including  federal  agencies,  of  electronic  instru- 
mentation (interpreted  in  the  broadest  sense)  in  order  to  have  them  identify  the  present  and  future 
critical  measurement  and  support  needs  in  this  field  -  physical  standards,  standardized  measure- 
ment methods,  new  calibration  and  measurement  assurance  services,  relevant  data,  and  most 
important,  new  measurement  methodologies. 

So  that  the  Workshop  would  be  conducted  in  a  manner  as  free  as  possible  from  any  biases  we 
in  the  Division  might  have,  and  to  ensure  that  segments  of  the  technical  community  generally  unfa- 
miliar with  the  Bureau  were  properly  reached,  the  services  of  Mr.  Peter  Richman,  a  well  known 
consulting  electronics  engineer  with  many  years  experience  in  electronic  instrumentation  and  a  long 
familiarity  with  NBS  and  especially  the  Electricity  Division;  were  retained  to  assist  in  planning  and 
conducting  the  Workshop.    Mr.  Richman  was  also  asked  to  prepare  the  final  report  on  the  Workshop, 
and  this  document  is  the  result.    Its  main  thrust,  as  we  believe  it  should  be,  is  technical  in  nature. 
That  is,  it  is  primarily  concerned  with  presenting  the  most  urgent  technical  needs  of  the  manufac- 
turing and  user  communities  in  the  field  of  dynamic  electrical  measurements  and  electronic 


instrumentation  as  identified  during  the  course  of  the  Workshop.    The  identification  of  the  field's 
most  important  problems  is  extremely  critical  if  the  Division  is  to  mount  an  effective  program  in 
such  a  broad  and  diverse  technology;   it  would  never  be  possible  for  the  Division  to  address  itself  to 
each  of  the  very  large  number  of  measurement  and  related  problems  which  exist.    Thus,  one  of  the 
main  goals  of  the  Workshop  was  to  provide  some  sense  of  priorities  and  to  delineate  those  most 
critical  needs  which  can  and  should  be  met  by  the  Electricity  Division.    We  therefore  look  to  the 
information  contained  in  the  report  as  basic  input  data  for  refining  and  further  developing  our  pro- 
gram.   Indeed,  it  is  probably  not  premature  to  state  that  the  technical  needs  identified  during  the 
Workshop  and  documented  here  will  form  the  corpus  of  the  Electricity  Division's  dynamic  electrical 
measurement  and  standards  work  in  support  of  modern  electronic  instrumentation  over  the  next  five 
to  ten  years. 

I  should  like  to  take  this  opportunity  to  personally  thank  all  of  the  Workshop  attendees,  and 
their  organizations,  for  making  the  time  available  in  their  busy  schedules  to  participate.    I  hope  each 
of  them  found  the  experience  to  be  as  rewarding  for  themselves  and  their  work  as  it  was  for  the 
Electricity  Division  and  its  staff.    I  also  wish  to  extend  a  special  thanks  to  the  four  Sub-Group  Chair- 
men for  their  extra  time  and  effort  and  for  a  job  well  done.     Finally,  I  should  like  to  acknowledge  the 
outstanding  work  of  Mr.  Peter  Richman  -  it  was  to  a  very  large  extent  his  involvement  in  the  plan- 
ning and  running  of  the  Workshop  which  made  it  so  successful. 

Barry  N.  Taylor,  Chief 
Electricity  Division 
Institute  for  Basic  Standards 
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Preface 

It  is  apparent  that  the  continued,  orderly  growth  of  the  electronics  industry  is  increasingly 
hampered  by  a  lack  of  standards  and  methodologies  in  a  number  of  crucially  important  electrical 
measurement  areas. 

The  explosive  growth  of  electronics  during  the  last  decade  has  brought  with  it  a  host  of  unsolved 
measurement  problems.    Yet  at  this  critical  historical  juncture,  electronics  is  a  key  technology  in 
the  prospective  solutions  to  some  of  society's  major  outstanding  concerns  including  energy,  pollu- 
tion, and  productivity  to  name  but  three.    It  is,  therefore,  mandatory  that  a  responsible,  capable 
organization  without  specific  industrial  affiliation,  assume  a  leadership  role  in  providing  the  neces- 
sary measurement  standards  and  methodologies,  as  well  as  in  establishing  new  electrical  measure- 
ment programs  for  which  need  is  both  broad  and  identified,  but  whose  implementation  by  specific 
industrial  organizations  would  be  impractical. 

The  Electricity  Division  of  the  National  Bureau  of  Standards  has  traditionally  dealt  with  the 
measurement  and  generation  of  electrical  signals  in  the  DC  to  30  kHz  range,  as  well  as  with  the 
properties  of  electrical  components  in  that  range.    In  effect,  with  the  one  possible  minor  (philosophi- 
cal) exception  of  phase,  the  Division  has  devoted  itself  to  amplitude  or  magnitude  measurements  for 
the  various  electrical  parameters*    Some  of  these  have  been  generated  and/or  transmuted  by 
mechanical  and  electro-mechanical  means,  others  more  recently  by  electronic,  and  others  in  future 
undoubtedly  by  light,  but  the  electrical  (usually  amplitude)  parameters  have  had  similar  properties 
and  required  the  same  disciplines  for  measurement. 

/Now,  automatic  systems  for  test,  process  control,  medical  applications  and  so.  forth  are  using 
electrical  quantities  at  higher  speeds  than  heretofore.    Ten  years  ago  a  precision  DC,  dialable  six- 
or  seven-digit  source  did  not  need  a  settling  time  specification;  today  "programmable"  sources  have 
such  specifications,  since  the  system  architect  must  know  how  long  it  will  take  for  this  stimulus  to 
settle  to  within  a  given  accuracy.    In  many  instances  total  system  speed,  hence  all-important 
throughput,  dollar  payback  and/or  justification,  will  be  limited  by  such  settling  times. 

Thus  to  precision  electrical  measurements  has  been  added  the  temporal  parameter.    In  a  man- 
ner analogous  to  the  changeover  from  frequency  to  time  domain  work  in  filter  synthesis  in  the  early  * 
sixties,  measurement  must  now  turn  from  exclusive  attention  in  the  frequency  domain,  to  charac- 
terization involving  time  as  well,  e.g.,  settling  time  to  x  microvolts  in  y  microseconds  is  an  elec- 
trical, analog,  amplitude  measurement  with  the  time  factor  added. 

The  Workshop  which  this  report  describes  and  summarizes,  is  the  means  chosen  by  the  Elec- 
tricity Division  to  establish  direct  communication  with  key  elements  of  the  electronics  industry 
among  both  manufacturers  and  users,  for  the  purpose  of  ascertaining  the  industry's  electrical  mea- 
surement needs,  both  general  and  specific.    The  enthusiasm  engendered  by  the  Workshop  itself 
attests  to  the  fact  that  the  time  has  come  for  leadership  and  action:    In  spite  of  difficult  times  and 
severe  cost  pressures,  twenty -four  outstanding  organizations  were  willing  to  send  top  personnel  to 
the  two-day  Workshop,  disrupting  busy  schedules,  and  at  considerable  financial  expense. 

This  report  on  the  Workshop  results  must  inevitably  reflect  some  of  the  author's  technical 
biases,  developed  over  the  course  of  twenty-five  years   of  varied  design,  management  and  consulting 
experience  in  the  fields  of  electronics,  measurements  and  standards.    During  that  interval  the  tech- 
nology has  made  tremendous  advances,  so  that  in  many  areas  it  is  virtually  unrecognizable  in  com- 
parison with  its  major  jumping-off  point  directly  following  the  Second  World  War.    Indeed,  some  of 
those  advances  are  the  direct  result  of  work  accomplished  at  or  backed  up  by  programs  of  the  Elec- 
tricity Division  of  the  NBS,  notably  during  what  might  be  termed  the  measurement  "explosion"  of  the 
decade  of  the  sixties. 
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By  the  same  token,  it  is  hoped  that  the  author's  general  breadth  of  design  and  measurement 
exposure  during  those  same  years  of  experience,  plus  long-standing  and  continuing  interactions  with 
the  Bureau,  have  provided  a  balanced  view  of  the  Workshop  results  in  this  report.    If  this  has  been 
accomplished,  the  Division  programs  resulting  from  the  Workshop  can  form  the  basis  for  new  and 
crucially-important  electrical  measurement  technology  for  electronics  in  the  forthcoming  decade. 

It  seems  clear  that  considerable  amounts  of  the  expertise  already  in  existence  in  the  Electric- 
ity Division  can  be  brought  to  bear  on  the  electrical  measurement  needs  of  modern  electronics,  as 
identified  in  the  Workshop  results.    This  is  particularly  true  in  the  areas  of  measurements  with  time 
as  an  added  parameter,  and  measurements  related  to  the  whole  field  of  automated  systems.    Over- 
poweringly  evident,  as  a  result  of  the  Workshop,  is  the  fact  that  an  entire  range  of  basic  unsolved 
problems  exist.    Also  clear  is  that  they  are  of  increasingly  critical  importance  to  continued  pro- 
gress, and  that  manufacturers  and  users  alike  look  to  the  Division  to  supply  the  necessarily  broad, 
technological  base  for  solutions. 

It  is  impossible  to  convey  here  the  excitement  felt  by  virtually  all  participants  during  the  Work- 
shop which  helped  to  make  the  author's  task  as  moderator  both  rewarding  and  enjoyable.    The  excite- 
ment was  undoubtedly  due  in  part  to  the  professional  challenge  offered  by  the  opportunity  for  open, 
free-swinging  exchange  among  the  outstanding  group  of  participating  engineers  and  scientists.    But  it 
clearly  reflected,  as  well,  the  participants'  recognition  of  the  large  and  important  work  that  so  crit- 
ically needs  to  be  done,  and  their  enthusiastic  reception  of  the  role  of  the  Electricity  Division  in 
grasping  the  initiative  to  establish  relevant  new  programs  on  a  priority  basis.    There  was,  in  short, 
the  excitement  resulting  from  the  pervasive  feeling  of  being  part  of  an  undertaking  that  was  both  sig- 
nificant and  effective. 

I  should  like  to  express  my  thanks  to  the  Electricity  Division  and  to  Dr.  Barry  N.  Taylor  in 
particular,  for  giving  me  the  opportunity  to  contribute  to  this  important  work.    I  should  also  like  to 
add  my  own  personal  thanks  to  the  participants  and  their  organizations,  as  it  is  they  who  are  truly 
responsible  for  the  Workshop's  success. 

Peter  Richman 
Consulting  Engineer 
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1.      Introduction 

1. 1     Background  and  Objectives 

The  traditional  role  of  the  Electricity  Division  of  the  National  Bureau  of  Standards  has  included 
a  variety  of  activities  directed  at  the  support  of  the  electrical  measurement  portion  of  the  National 
Measurement  System.    However,  as  industrial  and  governmental  applications  of  the  newer  electronic 
technologies  have  proliferated,  it  has  become  apparent  that  the  character  of  the  Division's  participa- 
tion -  indeed  leadership  -  must  further  evolve  if  it  is  to  continue  responsively  to  serve  the  nation's 
electrical  measurement  needs,  specifically  in  this  critical  new  arena. 

The  pervasiveness  of  electronic  technology  has  brought  to  the  industrial  production  floor  a 
measurement  sophistication  in  some  traditional  areas  that  rivals  what  would  have  been  considered 
outstanding  for  a  well-equipped  laboratory  not  too  many  years  ago.    Yet  the  measurement  philosophy 
exemplified  by  careful  theoretical  identification  of  all  sources  of  uncertainty,  followed  by  equally 
careful  experimental  verification,  can  hardly  be  said  to  have  accompanied  this  sophistication  to  its 
new  location.    The  Measurement  Assurance  Programs  (MAP's)  with  which  the  Electricity  Division  has 
become  increasingly  concerned  of  late,  are  intended  in  part  to  be  a  first  step  in  providing  a  higher 
degree  of  measurement  assurance  in  situ  -  as  close  to  the  measurement  site  as  possible. 

While  moving  forward  to  assist  in  bringing  measurement  integrity  out  of  the  standards  labora- 
tory and  into  the  field,  the  Division  has  become  aware  that  electronics  technology  -  a  tool  for  users  - 
has  wrought  a  basic  change  in  the  kind  of  measurements  that  delimit  continued  industrial  growth,  and 
another  in  the  environment  in  which  those  measurements  must  be  made.    It  was  to  more  clearly 
define  the  implications  of  these  changes  in  the  context  of  the  Division's  potential  contributions,  that 
the  idea  of  a  workshop  entitled,  "Critical  Electrical  Measurement  Needs  and  Standards  for  Modern 
Electronic  Instrumentation",  was  conceived. 

The  overall  Workshop  objectives,  then,  included: 

1.  To  ascertain  whether  there  are  significant  electrical  measurement  needs  in  the  exploding 
electronic s/instrumentation  industrie s. 

2.  To  determine  the  degree  of  universality  of  the  needs  as  identified. 

3.  To  investigate  whether  these  identified  needs  represent  areas  in  which  the  Electricity 
Division  of  NBS,  with  its  extensive  electrical  background  and  expertise,  could  contribute 
or  lead. 

4.  To  identify  broad  areas  of  electrical  measurement  need  of  sufficient  importance  that 
solutions  would  represent  major  contributions,  yet  not  so  state-of-the-art  as  to  prevent 
realistic  programs  from  being  structured  at  this  time. 

5.  To  further  identify  specific  project -level  needs  within  the  broad  program  definitions  and 
to  establish  realistic  bases  for  projects  that  can  be  undertaken  in  real-time  with  spe- 
cific, identifiable  impact. 

6.  To  ascertain  the  general  climate  of  opinion  regarding  longer  term  implications  of  elec- 
tronics to  electrical  measurements,  as  the  basis  for  strategic  planning  and  as  an  aware- 
ness input  for  future  programming. 

7.  To  establish  effective  new  lines  of  communication  between  the  Division  and  a  new  con- 
stituency represented  by  both  manufacturers  and  users  within  the  burgeoning  electronics 
industry. 


1.2     Participants  and  Industry  Distribution 

To  be  effective,  the  Workshop  was  to  be  small,  with  not  more  than  twenty-five  participants. 
To  provide  breadth,  the  participating  organizations  would  of  necessity  be  diverse.    Finally,  to  obtain 
depth  of  coverage  in  diverse  fields  with  a  small  group,  individual  participants  would  have  to  be  the 
most  knowledgeable  available  in  their  specific  fields. 

After  preliminary  studies  identified  broad  technical  areas  of  potential  interest,  organizations 
representative  of  each  of  these  areas  were  selected  with  some  attempt  to  balance  the  often  conflicting 
requirements  of  diversity  in  size  and  geographical  location,  the  decision  to  limit  participation  to  a 
single  attendee  per  organization,  and  an  organization's  known  or  supposed  interest  in  measurement 
technology  and  integrity  per  se.    With  regard  to  the  last  point,  some  organizations  were  specifically 
chosen  because  of  their  previous  apparent  disinclination  to  participate  in  the  activities  of  the  mea- 
surement community,  although  they  enjoy  leadership  positions  in  their  respective  fields. 

Last  but  not  least,  organizations  were  selected  for  their  known  or  expressed  willingness  to 
make  available  to  the  Workshop  the  time  of  busy,  top  technical  executives  who  retain  significant  day- 
to-day  technical  involvement  in  their  organizations'  activities.    The  Division's  success  in  achieving 
this  last  objective  is  attested  to  not  only  by  the  Workshop  results,  but  directly  by  the  participants 
themselves.    There  was  almost  unanimous  agreement  that  many  important  benefits  accrued  as  a 
result  of  the  opportunity  for  rewarding,  high-level  technical  interchange  with  peers. 

Diversity  then  was  the  objective,  with  depth  provided  by  the  quality  of  organization/participant 
selection.    Inevitably,  many  compromises  were  made,  fine  organizations  omitted,  and  numerous 
outstanding  individuals  not  included,  because  of  the  limitations  imposed  by  the  necessity  of  keeping 
the  group  to  a  reasonable  working  size.    It  is  hoped  that  when  participant  recommendations  are 
implemented  and  future  workshops  held,  some  of  these  necessary  omissions  can  be  rectified  by 
reconstituting  the  group  along  different  lines. 

1.3     Workshop  Agenda  and  Organization 

It  was  recognized  that  the  topical  agenda  would  critically  determine  the  results  of  the  Work- 
shop in  every  regard.    If  topics  for  discussion  were  not  identified  at  the  outset,  there  was  a  signifi- 
cant risk  that  the  resulting  agenda  would  be  so  undefined  that  it  would  generate  little  interest  and 
enthusiasm  among  potential  participants.    Still  worse,  a  loosely -organized  topical  structure  could 
result  in  a  mass  of  generalities,  useless  in  obtaining  the  necessary  overview  of  the  state  of 
electronics- related  electrical  measurements,  let  alone  an  indication  of  specific  programs  and  proj- 
ects. 

On  the  other  hand,  it  was  equally  clear  that  too  specific  and  too  rigid  a  topical  agenda  could 
well  stifle  the  independent  outside  thinking  that  was  the  primary  raison  d'etre  for  the  Workshop  in  the 
first  place. 

Last,  it  was  appreciated  that  if  the  agenda  was  to  appeal  to  representatives  of  an  electronics 
industry  already  divided  primarily  along  lines  related  to  corporate  and  other  organizational  inter- 
ests, it  must  to  some  extent  reflect  those  existing  divisions. 

The  topical  agenda  originally  consisted  of  three  main  sub-groups,  based  primarily  on  indus- 
trial divisions  rather  than  on  technology. 

1.        Systems  for  Automatic  Measurement  and  Control 

Includes  automatic  test  equipment  (ATE's)  and  systems  for  industrial  process  control. 


2.  Instruments  and  Functional  Modules 

Under  instruments  may  be  included  digital  voltmeters  (DVM's),  digital  multimeters 
(DMM's),  and  sampling  voltmeters;  programmable  DC  and  AC  voltage  and  current  sup- 
plies; calibrators  and  references;  self -balancing  impedance  bridges;  digital  phase  angle 
meters  and  standards;  wattmeters;  noise  generators;  scanners  and  multiplexers;  and 
filters. 

Under  functional  modules  may  be  included  analog-to -digital  and  digital -to -analog  con- 
verters (ADC's  and  DAC's);  sample- and -hold  amplifiers;  multipliers  and  dividers; 
instrumentation  and  operational  amplifiers;  comparators;  voltage-to-frequency  and 
frequency-to-voltage  converters;  AC-to-DC  and  DC-to-AC  converters;  and  filters. 

3.  Components 

Included  here  are  solid  state  voltage  references;  precision  resistors  and  networks;  pre- 
cision capacitors;  and  switches  and  relays. 

From  this  selection  of  topics,  it  would  be  apparent  to  potential  Workshop  attendees  that  to 
examine  the  problems  associated  with  measuring  the  classic  quantities  of  voltage,  current  and 
impedance  in  very  short  periods  of  time,  that  is,  electrical  measurements  in  the  time  domain,  could 
well  serve  as  a  concise  statement  of  the  Workshop's  major  aim. 

It  was  soon  recognized  that  some  technological  distinctions  among  the  three  topical  areas 
might  be  desirable,  and  a  more  detailed  sub-grouping  was  considered. 

1.  Systems  (e.g.,  ATE' s) 

2.  Signal  Conditioning  (e.g. ,  multiplexers,  sample-and-hold  amplifiers) 

3.  Sources  and  References  (e.g.,  programmable  supplies,  solid  state  references) 

4.  Data  Conversion  (e.g.,  ADC's,  DVM's) 

5.  Automated  Instruments  (e.g. ,  impedance  bridges) 

Prospective  Workshop  participants  were  contacted  with  the  general  Workshop  outlook, 
approach  and  objectives.    After  tentatively  agreeing  to  participate,  they  were  asked  to  submit  a  one- 
or  two-page,  pre-Workshop  write-up  of  their  most  critical  measurement  needs  and  problem  areas. 
It  was  recognized,  and  participants  were  so  informed,  that  these  pre-Workshop  inputs  would  play  a 
crucial  role  in  shaping  the  final  agenda  and  the  Workshop  organization,  and  would  independently  con- 
stitute an  important  Workshop  result. 

The  response  to  the  request  for  pre-Workshop  input  was  broad-ranging  and,  in  many  cases, 
quite  detailed.    It  is  given  in  section  A.  1  of  this  report.    As  anticipated,  the  inputs  were  of  great 
importance  in  helping  to  structure  the  Workshop  agenda  and  sub-groupings.    On  this  basis,  the  final 
topics  became: 

I.  Components  and  Parameter  Measurements 

IL  Signal  Conditioning  and  Data  Conversion 

III.  Sources  and  References 

IV.  Systems 

Table  1  summarizes  in  short  form  the  pre-Workshop  written  input  and  gives  some  idea  of  the 
specific  subjects  covered  by  each  of  these  four  major  sub-groups.    Note  that  for  each,  two  roughly 
different  classes  of  information  are  listed:    To  the  left  are  the  application  areas,  and  to  the  right  the 
required  technologies. 


Table  1.     Workshop  topics  based  on  the  pre-Workshop 
written  input  of  the  prospective  attendees. 


I.      COMPONENTS  AND  PARAMETER 
MEASUREMENTS 


Components 
Capacitors 
Resistors 

R-2R  Ladders 

Attenuators 
Switches 
Relays 

Parameter  Measurements 
I,  V,  Z 


Power 
Phase 


Accuracy  vs.  frequency 

Temperature  coefficient 

Settling  times 
Dielectric  absorption 

Computer  aided/enabled 


II.      SIGNAL  CONDITIONING  AND  DATA 
CONVERSION 


Attenuators 
Switches,  Relays 
Multiplexers 
Amplifiers 

Sample  &  Hold 

Operational 
Comparators 
Multipliers 

Other  Operators  (log,  etc. 
Converters 

Analog  to  Digital 

Digital  to  Analog 
Digital  Voltmeters 


Settling  times 
Thermal 
Electrical 

Acquisition  times 


Stabilities 
Long-term 
Short-term 

Temperature  effects 

Noise 
Self- generated 
Immunity  from 


IE.      SOURCES  AND  REFERENCES 


Solid  State  DC 
Voltage  References 
(all  levels) 

Primary  Cells  and 
Batteries 


Generators  and  Mea- 
surement Apparatus 


Sampled  Data 
Impulse  Testing 


Stability 
Noise 

Self-generated 

Immunity  from 
Transportability 
Reliability 

Environmental  effects 
Lifetime 

DC  voltage  and  current 
AC  voltage  and  current 

Sinusoidal,  ramps 

Pulses,  other 
Noise,  other  factors 

as  above 


IV.      SYSTEMS 

Time  as  a  Parameter 

Noise  as  a  System 
Parameter 

Systems  Problems 
Calibration  • 


Environment 


Software 


How  accurate,  how  fast 

Generation  and 
immunity 

Grounding,  interac- 
tion, interfacing, 
switching,  etc. 

In  situ  vs.  transport- 
ing, self- 
calibration,  opti- 
mum intervals 

Temperature,  rela- 
tive humidity 

Program  aided/ 
enabled  measure- 
ments 

Program  library 

Applications  pro- 
grams 

High  level  executives 


Organization  of  the  Workshop  presented  the  challenge  of  providing  sufficient  structure  to  ensure 
adequate  focus  for  the  generation  of  both  effective  discussions  and  results,  yet  within  a  form  free 
enough  to  provide  openness  for  unplanned  inputs.    Thus,  the  final  structure  included  a  mix  of  plenary 
sessions  involving  all  participants  simultaneously,  and  smaller,  working-group  or  sub-group  meet- 
ings organized  along  lines  identical  to  the  final  four  topical  areas  given  above. 

The  complete  agenda  for  the  two-day  Workshop  is  given  in  Appendix  B.    The  first  meeting  was  a 
plenary  session  held  the  morning  of  the  first  day.    Its  purposes  were  (a)  to  discuss  the  pre-Workshop 
input  and  any  changes  that  might  be  suggested  in  the  organization  of  the  remainder  of  the  Workshop; 
and  (b)  to  hold  short,  "mini"  sub-group  meetings  to  insure  that  all  participants  were  thinking  along 
the  same  lines.    Since  the  larger  group  was  to  be  split  into  the  four  working  groups  that  afternoon,  it 
was  deemed  important  to  obtain  simultaneous  exposure  of  all  participants  to  the  lead  concepts  and 
general  approach  in  each  of  the  four  working-group  areas.    This  was  considered  particularly  signifi- 
cant in  view  of  the  fact  that  only  two  working-group  sessions  were  planned,  with  all  four  working 
groups  meeting  simultaneously  each  time  -  once  that  afternoon  and  again  the  following  morning. 
Thus,  participants  would  be  faced  with  a  choice  of  attending  two  out  of  the  four  working-group  meet- 
ings.   (In  point  of  fact,  assignments  to  working-group  sessions  the  first  afternoon  had  been  made  in 
advance,  while  assignments  for  the  next  morning  were  completely  open  to  the  participants,  with  the 
possibility  of  attending  two  or  even  more  of  the  working-group  meetings  during  the  second  session. ) 

The  working-group  chairmen,  four  participants  who  kindly  agreed  to  serve  in  this  important 
capacity,  remained  with  their  groups  during  both  sessions  in  order  to  provide  needed  continuity. 

Between  the  two  working-group  sessions  there  was  a  short,  plenary  feedback  meeting  to  start 
the  morning  of  the  second  day.    This  provided  the  opportunity  for  an  exchange  of  views  on  progress, 
problems  and  possible  schedule  and  program  changes.    Out  of  it  grew  the  suggestion  that  partici- 
pants who  chose  to  do  so,  could  elect  to  visit  a  number  of  working-group  meetings  during  the  second 
session. 

Following  the  second  working-group  session,  the  final  plenary  meeting  was  held  the  second 
afternoon.    It  consisted  mainly  of  individual  summary  reports  presented  by  the  working-group  chair- 
men, followed  by  open  discussion. 

Electricity  Division  personnel  participated  in  both  the  plenary  and  working-group  meetings. 
The  composition  of  the  various  groups  for  both  working-group  sessions,  including  the  Electricity 
Division  personnel,  is  given  in  Appendix  B.    Bureau  personnel  participated  both  as  observers  and  as 
direct  participants  at  all  sessions,  and  provided  highly  skilled,  necessary  service  as  recorders  at 
the  working-group  meetings. 

Bureau  participants  and  recorders  worked  with  the  group  chairmen  to  prepare  both  the  verbal 
reports  for  the  final  plenary  session,  given  in  section  A.  4.,  and  the  written  reports  incorporated 
in  section  A.  3. 

The  generally  high  level  of  interest  and  commitment  during  the  Workshop  resulted  in  numerous 
small  and  large  group  interchanges  during  breaks  and  at  mealtimes  throughout  the  two-day  period. 
These  often  resulted  in  valuable  contributions,  and  helped  continue  the  forward  momentum  gained 
during  the  scheduled  sessions.    Even  these  "formal"  meetings  were  held  in  an  informal  manner, 
thereby  allowing  free  and  open  discussion  and  -  at  times  -  disagreements  that  eventually  led  to  the 
productive  results  and  insights  recorded  in  this  report. 

At  the  Workshop's  conclusion,  participants  filled  out  individual  "results"  and  "Workshop  com- 
mentary" sheets.    This  provided  them  a  final  opportunity  to  express  their  purely  individualistic 
viewpoints.    Summaries  of  these  comments  appear  in  sections  A.  5.  and  A.  6. 


1.4     Data  Sources  and  Data  Reduction 

A  total  of  six  separate  Workshop  data  inputs  provide  the  basis  for  the  results  and  conclusions 
embodied  in  section  2.  of  this  report.    These  data  appear  in  Appendix  A,  Data  Sources.    Although 
every  attempt  has  been  made  to  ensure  that  section  2.  faithfully  summarizes  the  information  con- 
tained in  the  data  sources,  it  was  felt  important  to  include  them,  even  at  the  risk  of  some  redun- 
dancy, so  that  the  reader  would  have  at  his  disposal  the  original  material  with  all  of  its  subtleties 
and  highly  personal  points  of  view.    However,  most  readers  probably  do  not  need  to  concern  them- 
selves with  Appendix  A  since  they  can  obtain  all  of  the  essential  ideas  from  section  2. 

The  data  sources  are: 

1.  Summary  of  Attendees'  Pre-Workshop  Input  (section  A.  1. ) 

As  already  indicated,  Workshop  attendees  were  requested  to  submit  pre-Workshop  let- 
ters identifying  critical  measurement  needs  as  they  saw  them.    The  information  from 
these  letters  appears  verbatim,  with  only  minor  editing  to  preserve  sense  when  not  sup- 
plied by  adjacent  context.    Entries  are  grouped,  however,  into  six  categories.    The  first 
four  are  identical  to  the  major  four  Workshop  sub -groupings.    The  last  two  contain  other 
inputs  not  directly  related  to  these  four  sub-groups. 

Electricity  Division  personnel  extracted  the  inputs  from  the  letters  and  grouped  them  by 
category,  with  some  assistance  from  the  moderator. 

2.  Notes  on  Opening  Plenary  Session  (section  A.  2. ) 

During  the  first  plenary  meeting,  a  general  exchange  of  views  took  place  and  four  mini- 
working-group  meetings  were  held.    Notes  on  these  discussions  provide  insight  into  the 
technical  orientation  of  the  participants  at  this  introductory  session,  and  give  an  indica- 
tion of  emphasis  useful  in  sorting  later  inputs. 

Notes  on  this  meeting  were  taken  and  edited  by  the  Workshop  moderator  for  inclusion  in 
this  report. 

3.  Working  Group  Reports  (section  A.  3. ) 

Sub-group  Chairmen,  along  with  the  Electricity  Division  (NBS)  participants  and  record- 
ers, remained  with  their  respective  working  groups  during  the  working  session  on  both 
days.    All  participated  in  the  preparation  of  the  working-group  reports,  which  constitute 
the  main  data  source  for  the  Workshop.    These  reports  were  summarized  verbally  by  the 
individual  group  chairmen  at  the  final  plenary  session,  as  already  noted. 

Each  sub-group  report  is  thus  the  result  of  a  cooperative  effort,  involving  the  associated 
sub-group  chairman,  NBS  recorder  and  NBS  participant.    General  editing  was  provided 
by  the  Workshop  moderator  for  organization  and  style  only,  with  virtually  no  modifica  - 
tion  to  content. 

4.  Notes  on  Sub-Group  Chairmen  Oral  Presentations  at  Closing  Plenary  Session,  with  Asso- 
ciated Discussion  (section  A.  4. ) 

Even  though  the  summaries  presented  by  the  chairmen  were  ostensibly  going  to  appear  in 
the  group  reports,  it  was  felt  important  for  several  reasons  to  capture  them  as  they  were 
presented.    First,  they  represent  reports  by  the  chairmen  themselves,  albeit  with  con- 
sultation with  the  associated  Bureau  personnel.    Second,  it  was  hoped  that  they  would 


convey  a  sense  of  the  tone  and  immediacy  of  the  discussions..  Both  of  these  assumptions 
appear  to  be  borne  out  by  the  results. 

The  entire  session  was  recorded  and  transcribed  by  a  professional  court  stenographer, 
with  editing  by  the  moderator  only  for  style. 

5.  Summary  of  Post -Workshop,  Written  Feedback  Reports  (section  A.  5.) 

As  indicated  previously,  at  the  termination  of  the  final  session  participants  were  asked  to 
fill  out  short,  individual-feedback  forms  giving  their  final  conclusions  regarding  critical 
measurement  needs  and  priorities.    These  free-form  inputs,  as  it  were,  presented  valu- 
able data  which  nevertheless  required  fairly  extensive  sorting  and  editing  by  the  moder- 
ator before  they  were  in  adequate  form  for  inclusion  in  this  report. 

6.  Summary  of  Post -Workshop,  Written  Commentaries  (section  A.  6. ) 

Also  as  previously  noted,  at  the  end  of  the  Workshop  participants  were  requested  to 
answer  a  short  questionnaire  about  the  Workshop  itself.    Answers,  again  sorted  and 
edited  by  the  moderator,  offer  insight  into  the  diversity  of  outlook,  the  degree  of  involve- 
ment of  the  participants  in  the  Workshop  itself,  and  attendee  hopes  and  expectations  for 
the  implementation  of  the  Workshop  results.    They  also  furnish  a  calibration  for  the 
degree  of  consensus  achieved  on  the  various  issues. 

Data  reduction,  or  final  extraction  of  program  and  project  information  considered  potentially 
useful  to  the  Electricity  Division  in  carrying  out  its  mission,  was  not  a  trivial  task  in  view  of  the 
nature  of  the  Workshop  and  the  plethora  of  input  data.    Significant  exclusions  from  the  final  summa- 
ries of  programs  and  projects  that  are  both  needed  and  real,  had  to  be  made  for  a  number  of  practi- 
cal reasons.    Areas  which  were  so  excluded,  and  covered  under  the  heading  "General  Long-Term 
Directions",  include: 

1.  Problems  with  solutions  which  seem  to  require  breakthroughs  either  not  presently  well 
foreshadowed  by  existing  or  anticipated  technology,  or  requiring  resources  well  beyond 
those  even  conceivably  available  to  the  Division.    Included  in  this  category  might  be 
replacement  of  the  standard  cell  as  the  ultimate  laboratory  working  standard  of  voltage, 
and  replacement  of  the  thermal  approach  to  precision  AC/DC  measurements. 

2.  Problems  whose  solutions  appear  to  have  limited  present  universality,  and/or  seem  to 
require  the  kinds  of  resources  -  and  motivation  -  more  likely  to  be  available  in  industry. 
Among  items  in  this  category  might  be  the  concept  of  impulse  testing  for  complex  sys- 
tems. 

3.  Problems  that  have  wide  universality,  but  no  presently-evident  "handle"  by  means  of 
which  any  single  group  of  investigators  would  be  likely  to  produce  results  of  significant 
value  to  any  large  part  of  the  field.    Included  in  this  category  might  be  the  general  prob- 
lems of  time-  and  history-dependence  of  systems  measurements,  with  the  exception  of 
those  aspects  that  can  be  quantified  in  terms  of  the  dynamic  performance  of  sources  and 
measurement  devices. 

Any  attempt  to  summarize  inputs  as  diverse  and  far-ranging  as  those  resulting  from  the  Work- 
shop must  of  necessity  involve  editing  and  the  application  of  judgment.    The  basic  criteria  for  mak- 
ing the  judgments  involved  in  defining  potential  programs  and  projects  were: 

1.  Breadth  of  application. 

2.  Relevance  to  the  mission  of  the  Electricity  Division. 


3.  Apparent  significance  in  terms  of  impact,  or  payoff  to  an  industry /government  constitu- 
ency. 

4.  Importance  in  terms  of  a  contribution  to  the  National  Measurement  System. 

5.  Implications  regarding  Electricity  Division  capability  to  move  forward  into  still  more 
sophisticated  areas  of  measurement  need. 

6.  Evidence  that  industry,  of  its  own  accord,  probably  cannot  or  will  not  deal  effectively 
with  the  problem  area. 

7.  Potential  for  reasonable  chance  of  success  in  a  reasonable  time,  versus  (3),  (4)  and  (5) 
above;  the  greater  the  apparent  impact,  the  larger  the  justifiable  program. 

8.  Potential  for  handling  via  reasonable  evolution  of  existing  Division  capabilities,  or  poten- 
tial for  sufficient  impact  to  justify  significant  rearrangement  of  priorities/capabilities/ 
facilities. 

In  light  of  the  above,  it  is  unlikely  that  the  conclusions  reached  will  generate  total  agreement 
in  any  single  quarter.    However,  it  is  hoped  that  they  will  generate  a  sufficient  degree  of  support 
among  a  wide  enough  constituency  to  ensure  that  many  of  the  Workshop  inputs  will  be  embodied  in 
meaningful  programs  of  the  Electricity  Division. 


2.     Results  and  Conclusions 

The  most  significant  conclusion  to  be  drawn  from  the  Workshop  is  that  there  is  a  strongly  felt 
need  on  the  part  of  the  electronics  industry  as  represented  for  guidance  in  electrical  measurement 
procedures  and  methodologies  on  a  broad  technological  front. 

Thus  the  Workshop  met  its  most  important  objective. 

Other  goals  were  met  as  well:   Universality  was  determined  for  specific  needs,  identification 
was  made  of  those  needs  that  can  be  met  by  expanded  programs  of  the  Electricity  Division,  and  both 
broad  programs  and  specific  projects  that  meet  the  requirements  just  listed  were  identified. 

In  addition,  the  more  general  objective  of  ascertaining  implications  for  the  longer  term  was 
also  achieved.    Results  are  incorporated  in  the  following  sections. 

Last  and  not  at  all  least,  a  significant  step  has  been  taken  towards  setting  up  new  lines  of  com- 
munication between  the  Division  and  a  far-flung  electronics- oriented  constituency. 

2. 1     Overview 

Three  categories  broadly  cover  the  needs  as  defined  by  the  Workshop: 

1.  The  need  to  introduce  time  as  a  measurement  parameter  has  resulted  from  the  require- 
ments of  automatic  test  and  control  systems.    Specific  areas  considered  critical  are: 

a.  Pulsed  component  measurements. 

b.  Dynamic  performance  characterization  for  modern  signal  conditioning  and  data  con- 
version: Digital-to-analog  and  analog-to -digital  converters,  sample -and -hold 
amplifiers,  comparators,  etc.    Required  measurements  include  settling,  aperture 
and  acquisition  times.    Basic  new  capabilities  will  be  needed,  including  precision, 
non-sinusoidal  waveform  generation  and  "standard"  digital-to-analog  converters. 

c.  Methodologies  and  techniques  for  characterizing  precision  AC  and  DC  sources  and 
measurement  devices  with  respect  to  settling  time. 

2.  The  emergence  of  measurements  into  the  "real- world"  of  the  production  line  via  the 
automatic  system  has  introduced  a  host  of  new  parameters.    Areas  requiring  significant 
effort  include: 

a.  Investigation  of  transportable  standards  for  validation  of  static  and  dynamic  system 
performance:  AC  and  DC  voltage,  impedance,  pulses,  settling  times. 

b.  Techniques  for  characterizing  sources  and  measurement  devices  with  respect  to 
switched  or  dynamic  loading. 

c.  Prediction  of  long-term  performance,  reliability  and  lifetime  from  short-term 
evaluation  for  a  host  of  passive  components,  semiconductors,  and  signal  sources. 

d.  Noise  standards  and  methodologies. 

e.  Continuing  effort  to  improve  transportable  DC  standards,  and  dissemination  at 
higher  voltage  levels. 

f.  Characterization  of  the  AC  line  voltage  -  waveforms,  under-  and  over -voltages, 
transients,  dynamic  impedance,  etc. 

g.  Inclusion  of  a  capability  for  environmental  control  and  variation  for  devices  and 
instruments  under  calibration,  in  all  new  work  undertaken  by  the  Bureau. 

h.      Extension  of  Measurement  Assurance  Programs  (MAP's),  especially  into  the  above 
new  areas. 


3.        Recent  work  at  the  leading  edge  of  measurement  technology  in  electronics  has  resulted  in 
the  need  for  new,  or  higher -accuracy  measurements.     For  example: 

a.  Increased  dependence  on  capacitors  as  storage  devices  has  made  dielectric  hystere- 
sis a  parameter  of  importance.    Significant  measurement  work  should  be  started  in 
this  area. 

b.  Phase  difference  measurements  to  extremely  high  accuracies  are  needed  to  calibrate 
industrial  instrumentation. 

c.  Higher  accuracy  power  measurements,  particularly  for  non- sinusoidal  waveforms, 
are  needed  to  calibrate  instrumentation  used  in  the  field. 

d.  Non- sinusoidal,  high  crest-factor  precision  signals  are  needed  to  calibrate  true  RMS 
converters  and  meters. 

e.  High  accuracy  electronic  instrumentation  requires  reduction  in  AC  calibration  uncer- 
tainties to  10  ppm,  at  least  over  the  audio  range. 

Longer-term,  "frontier"  areas  in  each  of  the  above  groups  were  also  identified.    They  are  not 
specified  in  the  above  listings  since  they  involve  work  at  still  more  basic  levels,  on  ground  that  is 
presently  terra  incognita.    Nevertheless,  they  furnish  insight  into  the  overall  technical/philosophic 
climate,  and  are  useful  for  that  reason.    Items  in  this  category  include: 

1.  Measurement  problems  in  systems,  including  effects  of  history,  measurement  interac- 
tions, self- calibration,  etc. 

2.  Evaluation  and  standardization  of  system  software. 

3.  Calibration  of  medical  equipment. 

4.  Replacement  of  standard  cells. 

5.  Replacement  of  thermal  AC/DC  converters. 

6.  Non-swept  but  broad -response  measurements  -  impulse,  noise,  other. 

7.  Basic  work  on  the  physics  of  dielectric  absorption;  component  drift;  device  noise;  failure 
mechanisms;  switches  and  relays. 

8.  Provision  of  improved  measurement  accuracy  for  virtually  all  dynamic  measurements,  at 
the  wafer  probe  level. 

Finally,  in  a  more  general  vein,  a  good  deal  of  support  surfaced  for  the  idea  of  keeping  the 
electronics  community  informed  of  Division  work-in-process,  with  emphasis  on  projects  targeted  by 
the  Workshop. 

2.2     Specific  Results 

The  preceding  section  outlined  Workshop  results  from  the  standpoint  of  technological  needs. 
This  section  presents  results  first  in  terms  of  broad  programs,  next  by  specifying  projects  falling 
within  these  programs,  and  finally  in  terms  of  some  general,  long-term  directions.    The  purpose, 
of  course,  is  to  begin  the  necessary  process  of  re -casting  needs  into  a  form  conducive  to  establish- 
ing a  coherent  set  of  programs  and,  where  possible,  specific  projects. 

2.2. 1     Broad  Program  Definitions 

This  section  attempts  to  formulate  responses  to  the  identified  technological  needs  in  program- 
matic form,  that  is,  in  groupings  that  share  a  common  technical/philosophical  background,  as  well 
as  potential  approaches  to  solution. 

In  accordance  with  this  approach,  broad,  needed  programs  that  can  be  readily  isolated  include: 

^        1.     Dynamic  Component  Measurements.      Characterization  and  measurement  of  the  pulsed 
characteristics  of  components  -  for  resistors,  capacitors,  networks;  also,  "conventional"  sine  wave 
measurements  at  higher  frequencies  than  now  available. 
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Passive  components  are  the  very  most  basic  elements  of  all  electronic  circuits,  instruments 
and  systems.  As  such,  along  with  their  active  counterparts,  they  are  responsible  for  the  ultimate 
performance  limits  for  electronic  assemblies. 

In  recent  years,  two  new  directions  have  become  apparent.    First,  components  are  being 
widely  used  in  critical,  switching-circuit,  data-conversion  applications.    Their  characteristics  in 
such  uses  -  settling  time,  hysteresis,  overshoot,  ringing,  etc.  -  have  been  all  but  ignored  by  classi- 
cal measurement  technologies.    Second,  as  the  electronics  field  advances  towards  still  faster  oper- 
ating speeds,  components  subjected  to  more  conventional  waveforms  like  sine  waves  are  being  used 
at  higher  frequencies.    This  trend,  too,  needs  to  be  reflected  in  the  upgrading  of  present  measure- 
ment methodologies,  in  this  case,  10  kHz  now,  100  kHz  and  higher  in  the  foreseeable  future. 

Impact  is  not  difficult  to  assess.    If  the  Bureau  does  not  move  to  characterize  components  on  a 
basis  that  reflects  industry's  real  needs,  then  the  notion  of  "traceability"  for  components  will  take  on 
an  increasingly  fictitious  significance.    Components  in  electronics  are  simply  not  used  the  way  they 
were;  the  Electricity  Division's  calibration  capability  for  components  must  reflect  this  if  that  capa- 
bility is  to  retain  relevance. 

2.  Building  Blocks  for  Signal  Conditioning  and  Data  Conversion.     Definition,  evaluation, 
characterization  of  dynamic  performance  for  modern  signal  conditioning  and  data  conversion  building 
blocks.    Initial  emphasis  on  digital -to -analog  and  analog-to -digital  converters,  sample -and -hold 
amplifiers,  comparators. 

An  entire  technology  has  grown  up  around  the  necessity  for  dealing  precisely  and  accurately 
with  the  interface  between  the  analog  and  digital  worlds,  and  manipulating  signals  on  the  analog  side 
of  the  interface.    This  far-reaching  technology  has  spawned  a  host  of  building  blocks  that  are  even 
appearing  in  integrated  circuit  form,  first  via  hybridization  and  eventually  via  monolithic  technology. 
In  all  forms  they  enjoy  widespread  application  as  basic  tools  for  data  manipulation  and  computing. 

There  is  a  significant  need  for  leadership  in  establishing  basic  measurement  methodologies  in 
this  new  field.    The  lack  of  wide  agreement  on  certain  critical  measurements  -  particularly  ampli- 
tude with  time  as  a  parameter  -  underlies  problems  in  the  standardization  of  terminology,  specifica- 
tions and  hardware. 

The  payback  from  the  rationalization  of  measurements  and  terminology  in  this  crucially- 
important  field  will  be  the  broader  application  of  the  existing  technology  to  a  wider  variety  of  indus- 
trial problems. 

3.  Dynamic  Measurements,  Sources  and  Measuring  Devices.     Methodologies  and  techniques 
for  characterizing  AC  and  DC  references  and  measurement  devices  with  respect  to  time  dependence, 
as  well  as  switched  or  dynamic  loading. 

Present  methods  for  calibrating  both  sources  and  measuring  devices  virtually  ignore  their 
dynamic  characteristics,  from  two  basic  standpoints. 

The  first  of  these  is  response  with  respect  to  time,  and  all  that  it  implies.    Included  is  not 
only  the  relatively  simple  notion  of  settling  time,  but  also  settling  time  for  various- sized  steps,  and 
in  particular,  steps  from  specific,  different  data  levels.    Also  of  significance  is  the  implication  of 
history  inherent  in  any  investigation  of  temporal  performance.    Included  is  response  from  overloads 
of  various  magnitudes,  response  for  steps  of  opposite  polarities  and  so  forth. 

The  second  dynamic  characteristic  that  often  leads  to  basic  performance  limitations  in  critical 
systems  is  response  to  dynamic  or  switched  loading.    This  factor  is  of  particular  importance  for 
sources.    Rapid  recovery  is  required  after  the  application  of  a  wide  variety  of  switched  loads, 
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including  the  direct,  instantaneous  short-circuits  resulting  from  multiplexer  make -before -break 
input  switching. 

4.  Static  and  Dynamic  Transportable  Standards.      Investigation  and  development  of  both  pas- 
sive and  active  transportable  standards,  suitable  for  in  situ  validation  of  automatic  system  perfor- 
mance:   Needed  are  standard  components,  sources,  measuring  devices. 

The  concept  of  taking  parts  of  a  system  to  a  calibration  facility  is  triply  inadequate:   They  may 
not  be  calibrated  when  they  return  due  to  changes  induced  by  shipping;  they  are  not  calibrated  under 
operating  environmental  conditions;  and  they  are  the  wrong  things  to  calibrate  anyway.    These  three 
factors  make  present  calibration  techniques  wholly  inadequate  for  the  bulk  of  the  electronic  instru- 
mentation systems  presently  planned,  and  already  in  the  field. 

There  would  seem  to  be  no  substitute  for  development  of  an  entirely  new  methodology  for  cali- 
bration of  modern  systems.    The  key  words  here  are:   Transportable  and  dynamic. 

First,  bringing  the  transportable  standards  to  the  system  insures  that  the  devices  and  systems 
being  calibrated  will  not  be  affected  by  shipment.    But  of  course,  the  standards  themselves  must  be 
designed  to  withstand  shipment.    Thus,  methods  would  have  to  be  employed  similar  to  those  used  for 
present  Division  MAP's  in  order  for  this  approach  to  be  successful. 

Next,  in  situ  calibration  results  in  the  closest  possible  approximation  to  actual  operating  con- 
ditions, environmental  and  otherwise. 

Last,  what  can  and  indeed  must  be  calibrated  at  the  site  are  things  which  are  unmeasurable  in 
the  laboratory:   Performance  of  the  entire  system,  not  just  its  building  blocks,  thereby  including  at 
least  to  some  extent,  history  and  interactions,  and  performance  at  the  system's  real -world  inter- 
face, not  simply  at  terminals  within. 

Also  mandatory  is  including  the  temporal  parameter  in  the  calibration  since  it  is  so  crucial  to 
system  performance  and  throughput.    Dynamic  standards  are  thus  required,  to  supplement  and  per- 
haps eventually  even  replace,  their  static  predecessors.    Needed  are  AC  and  DC  sources,  resistors 
and  capacitors,  plus  devices  for  voltage  and  current  measuring  and  monitoring.    Critical  for  all 
such  dynamic  standards  is  precise  knowledge  of  their  performance  as  a  function  of  time  after  they 
are  switched,  energized  or  activated. 

Of  all  the  programs  identified  by  the  Workshop,  there  was  the  clearest  consensus  that  this  is 
undoubtedly  one  of  the  most  significant  for  the  near  term.  It  represents  a  pervasive  need  and  from 
many  standpoints,  it  constitutes  perhaps  the  greatest  immediate- challenge. 

5.  Predictions  for  Component  Stability.      Estimates  of  long-term  performance,  reliability 
and  lifetime  from  short-term  evaluation,  for  the  full  range  of  passive  and  active  components: 
Resistors  (thick  and  thin  film),  networks,  capacitors,  semiconductors.    Efforts  would  presumably 
start  with  extensive  component  stability  and  reliability  studies. 

Again,  components  are  the  ultimate  bits  and  pieces  whose  performance  limits  that  of  modules, 
instruments  and  systems.    Thus,  knowledge  of  their  basic  capabilities  is  crucial  to  calculating  char- 
acteristics of  the  larger  assemblies  that  depend  on  them.    This  is  nowhere  more  valid  than  in  the 
area  of  performance  stability.    The  electronics  designer  can  often  obtain  circuit  or  device  charac- 
teristics with  a  high  degree  of  independence  from  changes  in  particular  components;  but  this  can  be 
costly,  and  in  any  event  there  are  always  some  few  components  in  any  assembly  that  determine 
overall  limits  to  performance. 

In  the  headlong  rush  to  apply  electronic  technology  to  the  variety  of  industrial  and  societal 
problems  for  which  it  holds  so  much  promise,  insufficient  knowledge  has  been  developed  regarding 
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the  causes,  character,  influences  on,  and  rates  and  magnitude  of  changes  in  the  entire  range  of  basic 
electronic  components,  both  passive  and  active.    And  while  some  of  the  work  that  would  clearly 
prove  useful  in  an  investigation  of  component  stability  is  so  basic  that  it  appears  in  a  later  section, 
it  would  seem  that  much  widely-useful  data-gathering  could  be  accomplished  without  getting  too  inti- 
mately involved  in  device  physics. 

Manufacturers  and  users  need  and  expect  equipment  to  be  stable,  reliable  and  long-lived. 
Component  derating  and  conservative  design,  in  the  hands  of  knowledgeable  designers,  has  produced 
equipment  that  often  succeeds  but  sometimes  fails  in  use,  as  a  result  of  problems  in  the  areas  of 
stability  and  reliability.    Much  work  in  this  regard  is  clearly  necessary  if  electronic  solutions  to 
medical,  transportation  and  production  problems,  to  name  but  a  few,  are  to  gain  widespread  accep- 
tance.   Only  a  central,  unbiased  organization  can  be  expected  to  carry  it  out  with  the  required  moti- 
vation and  impact. 

6.  Energy-Source  Status.     Determination  of  long-term  battery  life,  charge  status  and 
rechargeability  from  short-term  measurements,  via  any  feasible  technique  including  battery 
implants. 

There  are  several  areas  in  which  a  prediction  of  battery  status  from  current  data  is  of 
importance.    The  first  is  the  obvious  medical  area,  in  the  burgeoning  field  of  electronic  implants. 
New  applications  besides  heart  pacing  are  proliferating,  and  performance  can  be  critical  to  life. 

A  second  application  for  battery  status  information  is  in  maintaining  standard  cell  or  other 
reference  standards  at  constant  temperature  during  transport.    Here,  emphasis  is  on  charge  status 
and  rechargeability.    This  will  constitute  an  increasingly  important  problem  as  the  trend  continues 
to  grow  towards  transportable  standards  and  MAP's. 

Battery -status  predictability  may  well  have  significant  additional  impact  on  the  whole  range  of 
battery-operated  appliances,  tools  and  equipment  used  in  both  the  home  and  industry:   Drills,  shav- 
ers, lawnmowers,  wire-wrap  tools,  and  others  the  designs  of  which  await  better -characterized 
energy  sources. 

7.  Noise.     Noise  standards,  measurement  methodologies,  characterization  of  distribu- 
tions, both  for  component  measurements  and  for  characterization  and  verification  of  noise  genera- 
tors. 

Noise  has  figured  in  classical  electrical  measurements  primarily  as  a  characteristic  to  be 
dealt  with  as  a  factor  limiting  the  minimum  uncertainty  that  can  be  attained.    But  the  broad  applica- 
tions area  of  noise  measurement  and  characterization  has  been  largely  bypassed  or  ignored. 

Meanwhile,  industry  has  been  struggling  with  noise  as  an  inherent  limit  to  passive  component 
performance  (resistors  of  several  varieties);  as  a  possible  indicator  of  stability  and  likewise  a  limit 
to  performance  (Zener  references);  as  an  unwanted  signal  in  systems  (crosstalk,  radiation  sensitiv- 
ity, ground  couplings);  and  as  a  restrictive  element  in  the  performance  of  a  wide  and  important 
range  of  semiconductors  (transistors,  operational  amplifiers).    Noise  generators  go  largely  uncali- 
brated,  and  non-gaussian,  semiconductor -generated  noise  distributions  remain  uncharacterized. 

Clearly,  noise  is  an  important  and  restraining  factor  in  much  work  in  electronics.    The  con- 
stituency of  sophisticated  designers  and  users  continues  to  expand.    As  it  does,  the  pressure  can 
only  mount  for  providing  the  same  fundamental  basis  for  noise  measurements  that  the  Electricity 
Division  has  helped  to  furnish  for  the  metrology  of  DC  and  periodic  AC  signals. 

8.  DC  Reference.      Continuing  effort  with  semiconductors,  possibly  including  new  work  with 
band-gap  devices,  towards  developing  1-5  ppm  stable,  transportable,  higher -voltage  DC  standards 
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with  predictable  drift  rates;  plus  continuing  effort  in  making  the  Josephson  voltage  standard  easier  to 
use. 

Virtually  all  instruments  and  systems  that  deal  with  signal  magnitudes  incorporate  at  least 
one  basic  DC  source  to  which  the  device's  generated  signal  and/or  measurements  are  referenced. 
The  stability  and  absolute  value  of  these  references  can  often  be  a  serious  limiting  factor  in  signal 
generation  and/or  measurement  validity,  with  significant  economic  implications  via  the  need  for 
shorter  or  longer  calibration  intervals,  not  to  mention  performance  itself. 

Both  industry  and  the  Bureau  have  done  a  great  deal  of  work  related  to  this  fundamental  refer- 
ence; but  much  more  needs  to  be  done,  e.g. ,  improve  the  ruggedness  of  semiconductor  references; 
disseminate  the  unit  of  voltage  at  higher  voltage  levels,  for  more  direct  traceability  at  lower  total 
uncertainties;  develop  transportable  standards  of  sufficiently  consistent  performance  to  allow  drift- 
rate  prediction;  and  improve  the  Josephson  standard  so  that  it  may  see  more  widespread  use. 

There  is  probably  no  more  basic  an  amplitude  reference  than  the  DC  reference.    The  Division 
has  a  long  history  of  maintaining  and  supporting  it,  when  its  use  was  primarily  for  reference  pur- 
poses within  a  standards  laboratory  environment.    However,  the  reference  has  now  moved  out  onto 
the  production  floor.    Division  support  must  move  with  it  in  order  to  ensure  continued  progress  in 
measurements  in  the  real-world,  systems  environment. 

9.  The  AC  Line.      Deterministic  and/ or  statistical  characterization  of  AC  line  voltage  at 
the  receptacle. 

The  AC  line  -  with  all  of  its  vagaries  -  supplies  primary  power  to  all  but  a  small  percentage 
of  the  electronic  equipment  in  common  use.    As  such,  its  peculiar  characteristics  and  the  design 
allowances  which  must  be  made  for  them,  have  direct  economic  impact. 

A  wide  range  of  parameters  urgently  need  rigorous,  even  if  statistical,  characterization,  in 
order  to  permit  designers  and  users  to  either  design  for  or  else  to  modify  the  AC  line  voltage  pres- 
ently available  at  the  receptacle.     Included  are  transients,  over-  and  under -voltages,  dynamic 
impedances  out  to  the  frequencies  and/or  rise  times  of  pulsed,   switched  and  switching  loads,  the 
effect  of  modern  electronic  loads  on  waveform  noise,  etc. 

In  this  area,  overdesign  typically  represents  wasted  effort  and  dollars,  while  underdesign  can 
result  in  expensive  outages  or  else  the  need  for  auxiliary  regulators  and  filters.    Evidently  signifi- 
cant economic  uncertainties  are  implied  by  the  present  inadequate  state  of  knowledge  and  agreement 
on  AC  line  parameters. 

10.  Data  Outliers.      Characterization  and  methodologies  to  measure  or  live  with  the  data 
outlier  problem  in  measurement  in  general,  and  systems  in  particular. 

In  any  series  of  measurements,  from  time  to  time  a  result  will  appear  that  is  so  vastly  dif- 
ferent from  its  neighbors  that  it  is  termed  a  data  outlier.    These  are  usually  discarded  in  "manual" 
measurements,  with  the  rationale  that  they  are  manifestations  of  unknown  source  of  noise,  most 
probably  external  to  the  measurement  complex  or  system. 

The  severity  of  the  outlier  problem  increases  with  measurement  speed,  since  upward  pres- 
sure on  speed  decreases  the  likelihood  that  measurements  can  be  employed  which  are  inherently  of 
the  averaging  type.    Sampling  an  input  parameter  during  an  interval  of  just  a  few  microseconds 
enhances  the  possibility  of  even  small  amounts  of  noise  causing  significant  errors. 

N         Another  factor  forces  attention  to  the  hitherto  neglected  outlier:    On-line  electronic  systems 
are  increasingly  taking  the  responsibility  for  monitoring  and  controlling  industrial  processes, 
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life-support  systems,  and  even  whole  micro-environments.    They  can  stand  fewer  mistakes,  since 
such  errors  can  have  adverse  human  and  economic  impact. 

Evidently  at  least  two  basic  philosophic  approaches  are  available  for  dealing  with  outliers  - 
reducing  their  incidence  via  a  program  of  measurement,  understanding,  and  finally  design,  or  sim- 
ply designing  instrumentation  on  a  basis  inherently  insensitive  to  them,  whatever  be  their  source. 
Leadership  and  knowledgeable  investigation  are  required  to  put  the  problem  in  perspective,  and  to 
delineate  realistic  alternatives. 

11.     Measurement  Assurance.      Extension  of  MAP' s,  specifically  into  the  new  areas  outlined 
in  the  above  programs;  some  effort  to  define  traceability,  implicitly  or  explicitly,  possibly  via  the 
MAP  route. 

Generating  the  required  new  methodologies  is  insufficient;  dissemination  is  necessary  as 
well,  and  in  a  way  that  inherently  guarantees  their  successful  application'to  the  solution  of  measure- 
ment problems  throughout  the  electronics  industry.    The  NBS  Measurement  Assurance  Program, 
extended  to  include  Workshop -defined  programs,  would  constitute  a  recognized  vehicle  for  success- 
fully accomplishing  this  objective. 

It  is  well  recognized  that  the  notion  of  traceability  to  the  Bureau  can  be  sorely  abused.    While 
it  is  doubtful  that  NBS  can  or  should  state  that  a  particular  industry,  or  worse  yet,  company,  is  mak- 
ing measurements  that  are  "traceable",  MAP's  in  critical  areas  could  play  an  important  role  in 
keeping  reasonable  bounds  on  use  of  this  important  concept.    And  the  industry,  as  represented  at  the 
Workshop,  would  welcome  efforts  to  help  "traceability"  retain  real  significance. 

2.2.2     Specific  Project  Activities 

While  there  will  be  much  additional  work  required  to  define  specific  projects  within  the  broad 
programs  outlined  in  the  previous  section,  cognizant  Workshop  participants  were  able  to  agree  on 
some  specific  activities  with  readily  apparent  and  quantifiable  goals,  and  immediate  payoffs  for 
industry: 

1.  Capacitors.      Capacitor  dielectric  hysteresis:    Evaluation,  characterization,  measure- 
ment, hopefully  leading  to  possible  improvements  via  subsequent,  basic  work  on  dielectrics. 

2.  Signal  Sources.     Theoretical  modeling  followed  by  the  design  and  construction  of  basic 
building  blocks  for  precision  dynamic  measurements: 

Pure  step  or  pulse  generator  -  20  ns  settling  time  to  at  least  0.  01%  accuracy,  to  be  used  for 
settling  time  and  other  measurements. 

Pure  ramp  and  ramp -with -corner  generator  -  100  ppm  linearity,  to  be  used  for  aperture  and 
acquisition  time  measurements. 

"Standard"  DAC  with  10  ppm  accuracy,  and  10  mA  output  capability.    Presumably  to  be  as 
"glitch-free"  as  possible,  and  with  a  settling  time  consistent  with  testing  devices  which  settle 
to  0.01%  in  20  ns. 

3.  Phase.     Phase  angle  calibration  capability  for  existing  commercial  instruments  claiming 
0. 01  degree  measurement  accuracy. 

4.  Power.      Power  calibration  capability  with  50  ppm,  if  not  20  ppm  uncertainty,  to  be  used 
in  the  calibration  of  commercial  0.  01%  and/ or  0. 02%  power  meters.    Increase  emphasis  on  non- 
sinusoidal  waveforms  for  both  power  and  power  factor  measurements  and  calibration. 
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5.  AC  Calibration,     Reduce  AC  calibration  uncertainties  to  10  ppm  in  the  audio  band. 

6.  High  Crest- Factor  AC.      Create  a  facility  for  calibrating  RMS-measuring  devices  for 
non-sinusoidal,  high  crest-factor  inputs,  with  crest  factors  to  10  or  12  at  full  scale. 

7.  Transportable  Standards.     Design  or  qualify  transportable  standards  for  in  situ  system 
calibration: 

DC  voltage  to  1000  V,  with  uncertainties  approaching  10  ppm. 

AC  voltage  to  1000  V  and  to  10  MHz,  with  midband  uncertainties  approaching  20  ppm. 

Resistance  from  0.  001  Q  to  100  MQ. ,  with  midband  accuracies  to  10  ppm. 

Pulse  standard  -  rise  times  of  10  to  30  ps,  exponential  decay. 

Constant  currents  (DC,  AC)  -  1  nA  to  10  A,  with  midband  uncertainties  to  50  ppm. 

2.2.3     General  Long-Term  Directions 

Lastly,  a  number  of  areas  were  identified  which  either  imply  programs  so  extensive,  or 
require  breakthroughs  so  apparently  distant,  that  they  cannot  be  included  in  the  "practical"  results 
sections  of  this  report.    However,  they  present  a  graphic  picture  of  the  philosophic  pressures 
underlying  electrical  measurement  needs  for  some  time  to  come.   As  such,  they  merit  study  not  only 
to  enhance  overall  awareness,  but  also  to  provide  the  basis  for  possible  future  program  definitions. 

Included  in  this  category  are: 

1.  System  Hardware.     Define  and  take  some  steps  towards  the  solution  of  the  problems  of 
verification  of  "system"  performance  in  the  field,  for  a  wide  variety  of  systems.    Definition  of  a 
real  focus  for  "transportable  dynamic  standards"  is  required,  for  the  most  general  class  of  sys- 
tems.   Measurement  assistance  for  dealing  with  time-  and  hi  story -dependent  effects  in  systems  is 
needed,  as  well  as  interconnects,  cablings,  grounds,  isolation,  guards  and  last  but  not  least,  noise 
generation  and/or  sensitivity.    System  self- calibration,  and/ or  automatic  out -of -calibration  indica- 
tions are  also  required.    Computer -controlled  calibrators/ sources  may  be  a  possible,  partial 
approach  to  some  of  these  problems. 

2.  System  Software.      System  software,  and  recommendations  on  preferred  approaches, 
are  fruitful  areas  for  long-term  effort. 

3.  Medical  Electronics.      Establishment  of  a  calibration  capability  for  medical  equipment, 
specifically  for  hospitals. 

4.  DC  Reference.      Replacement  of  standard  cells  with  another  reference  of  equal  stability, 
but  far  greater  environmental  in  sensitivity.    Preferred  is  the  10-volt  level,  with  a  10  C  to  50  C 
operating  range,  with  a  temperature  coefficient  small  enough  to  be  consistent  with  an  overall  stabil- 
ity of  better  than  one  ppm.    Portability  is  important,  although  being  self -powered  is  not.    An  output 
capability  of  up  to  10  mA  is  desired.    Warmup  of  up  to  an  hour  can  be  tolerated. 

5.  AC  Measurement  or  Transfer.    Develop  an  RMS  measuring  device  (to  replace  AC/DC 
thermal  transfer  standards  as  we  now  know  them),  with  accuracies  approaching  10  ppm  (100  ppm 
satisfactory  for  now),  but  more  important,  capable  of  operating  at  higher  speeds.    Ten  measure- 
ments per  second  are  needed  now,  and  up  to  100  per  second  in  the  future.    Its  dynamic  range  should 
be  far  greater  than  present  thermal  devices  -  up  to  at  least  10:1  for  voltage;  and  the  device  should 
be  rugged  and  relatively  insensitive  to  overload. 
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6.  Non-Swept,  Broad  Response  Tests.      Impulse  and/or  noise  testing  criteria  would  be 
desirable,  plus  methodologies  and  hardware.    Areas  of  use  would  include  application  to  non-swept 
but  broad  response  tests  for  complex  devices  and  systems  (see  wafer  probing  below),  with  the 
resulting  data  analyzed  for  correspondence  with  present,  more  conventional,  "manual"  methods. 
Fast  Fourier  transform  techniques  might  be  a  possible  approach. 

7.  Dielectric  Absorption.    Basic  work  is  needed  on  the  physics  of  dielectric  absorption, 
with  a  view  towards  improving  performance  of  compact,  solid -dielectric  capacitors. 

8.  Component  Drift.    Basic  work  is  needed  on  the  physics  of  component  drift:    Film  resis- 
tors, bipolar  transistor  VR    's  and  Zener  references  seem  most  important.    Included  would  be  sta- 
bility of  device  temperature  coefficients,  for  both  passive  and  active  devices.    Object  would  be  to 
reduce  component  drift  and  to  predict  future  values. 

9.  Component  Noise.     Same  as  (8)  for  device  noise,  which  is  presently  becoming  a  final 
performance  limit  for  both  resistors  and  semiconductors. 

10.  Component  Failure.     Basic  work  is  needed  on  the  physics  of  failure  mechanisms  for 
both  passive  and  active  components,  including  contact  devices  like  relays  and  switches. 

11.  Relays  and  Switches.     Basic  work  is  required  on  the  theoretical  and  experimental 
design  of  advanced  relays  and  switches  for  system  applications.    Goal  would  be  high  reliability  with 
low  contact  resistance  and  thermals,  and  a  high  voltage  handling  capability.    High  accuracy  switch- 
ing is  especially  needed  for  AC  voltage  and  currents  up  to  10  MHz. 

12.  Wafer  Probe  Testing.      Last,  there  is  a  foreseeable  need  to  provide  means  for  virtually 
all  high  speed  electrical  measurements  at  the  wafer  probe  level,  taking  into  account  length  of  leads, 
noise,  etc. 

2.3     Conclusions 

The  Workshop  and  its  results  as  summarized  here  are  but  the  first  steps.    Broad  programs 
have  been  outlined  and  specific  projects  identified.    Priorities  must  now  be  assigned,  however,  if  an 
organization  of  finite  size  is  to  attack  even  a  reasonable  number  of  the  problem  areas  delineated.    In 
effect,  the  industry  is  perhaps  a  decade  ahead  of  the  metrology  it  needs  to  continue  its  growth;  but 
this  gap  cannot  be  narrowed  overnight,  and  wise  choices  must  be  made  concerning  alternative 
courses  of  action. 

It  is  apparent  that  the  electronics  industry  has  generated  a  host  of  electrical  measurement 
problems  different  in  kind  from  those  of  classical  metrology.    Now  needed  are  impact  studies,  pri- 
ority assessments  and  in-depth  evaluation  for  all  potential  program  areas  as  identified.    Also 
required  is  reevaluation  of  ongoing  Division  programs  in  the  light  of  the  Workshop  results. 

One  final  conclusion  remains:    It  was  the  overwhelming  consensus  of  the  attendees  that  the 
Workshop  should  be  repeated  in  one  form  or  another  at  intervals  of  one  to  two  years.    This  kind  of 
participant  interest  and  commitment  implies  continuing  cooperation  and  maximum  impact,  as  the 
new  programs  are  implemented. 
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Appendix  A:     Data  Sources 

A.  1     Summary  of  Attendees'  Pre-Workshop  Input 

The  following  six  sections  are  appropriately  grouped  excerpts  from  the  pre -Workshop  written 
input  submitted  by  the  attendees.  Only  minor  editing  was  carried  out  by  the  moderator  and  Electric- 
ity Division  staff.  At  the  beginning  of  the  first  four  sections  -  Components  and  Parameter  Measure- 
ments; Signal  Conditioning  and  Data  Conversion;  Sources  and  References;  and  Systems  -  two  side-by- 
side  short  lists  of  words  and/or  phrases  are  given  which  attempt  to  capture  the  essence  of  the  topic 
under  discussion.  Roughly  speaking,  the  left-hand  list  gives  the  application  areas  and  the  right-hand 
list  the  required  technologies. 

A.  1. 1     Components  and  Parameter  Measurements 

Components 

Capacitors  Accuracy  vs.  Frequency 

Resistors 

R-2R  Ladders  Temperature  Coefficient 

Attenuators 
Switches  Settling  Times 


Relays 


Dielectric  Absorption 


Parameter  Measurements 
I,  V,  Z 

Power  Computer  Aided/ Enabled 

Phase 

1.  Dielectric  absorption  in  capacitors  is  a  limiting  factor  in  the  performance  of  sample- 
and-holds  and  integrating  DVM's.    We  need  dielectric  absorption  specifications  that  can  be  meaning- 
fully related  to  performance  in  the  above  instrumentation.    Percent  dielectric  absorption  is  very 
much  related  to  the  test  method,  and  the  frequency  dependence  of  loss  tangent  at  low  frequencies  is 
almost  useless  in  predicting  performance  in  a  sample -and -hold. 

2.  For  AC  standards,  such  as  capacitors,  better  standards  over  a  much  wider  range  of  fre- 
quencies are  expected  to  be  needed. 

3.  All  methods  of  measurement  of  physical  circuit  parameters  and  constants,  particularly 
in-circuit  values  of  R,  C,  L,  etc. 

4.  In  the  component  area,  the  advent  of  stable  film  resistors  has  lessened  the  demand  for 
wirewound  resistors.    They  are  still  used,  however,  and  standard  settling  time  tests  for  wirewound 
resistors  would  be  desirable. 

5.  As  we  depart  from  the  primary  basic  electrical  standards,  we  also  face  limitations. 
Practical  use  of  resistance  from  10    ohms  to  10      ohms  is  limited  by  the  high  voltage  coefficient  of 
the  resistors. 

6.  Two  standards  areas  of  definite  interest  are  high  resistance  and  low  voltage  measure- 
ments.   Our  5155  sets  of  hi-meg  resistors  (10    to  10      ohm)  are  regularly  sent  to  the  Bureau  for 
measurement.    We  find  that  NRC  (Canada)  offers  hi-meg  calibration  to  less  uncertainty  than  NBS  (at 
109,  0.05  vs.  0.2%  and  at  1012,  0.35  vs.  0.5%).    Desirable  in  the  near  future  would  be  an  order  of 
magnitude  improvement  in  the  Bureau's  uncertainty. 

7.  A  calibration  standard  for  ladder  networks  of  the  R-2R  type  is  needed. 
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8.  High  reliability  relay  with  very  low  contact  resistance  upon  contact  closure:   Because  of 
the  large  number  of  relays  in  a  system  (and  the  requirement  of  contact  resistance  to  be  0.080  ohm 
maximum),  the  closed  contact  resistance  shall  not  exceed  0.080  ohms  more  than  once  m  10,  000 
operations. 

9.  We  also  see  a  need  for  smaller  uncertainties  in  the  calibration  of  AC  devices  through  all 
frequencies,  but  especially  up  through  10  MHz.    Repeatability  of  measurements  in  this  range  is  often 
better  than  the  uncertainty  in  many  cases.    This  problem  extends  itself  into  the  precision  attenuator 
calibration  accuracies  too.    Requests  have  been  made  for  0.  001  dB  at  -90  dB. 

10.  Required  are  precision  measurements  of  very  small  currents  and  the  ratios  of  very 
small  currents,  say  in  the  below  10"    ampere  region;  also  as  a  corollary,  the  precision  measure- 
ment of  very  high  impedances. 

11.  Low  voltage  calibration  (10  nanovolts  to  microvolts)  is  done  using  SR1010  sets  to  cali- 
brate precision  four-terminal  low-thermal  dividers.    This  is  a  time-consuming  and  expensive  task 
and  nearly  impossible  for  most  of  our  customers.    Currently,  1  millivolt  (to  100  nanovolts  accu- 
racy) and  100  microvolts  (to  10  nanovolts  accuracy)  are  the  most  sensitive  requirements.    I  see 
future  needs  for  about  an  order  of  magnitude  uncertainty  improvement. 

12.  What  is  the  state-of-the-art  on  AC  standards  with  respect  to  high  input  impedance? 

13.  The  variable  to  be  measured  is  generally  very  slowly  varying  (less  than  0. 1  Hz)  and  is 
derived  from  a  transducer  such  as  a  thermistor  or  thermocouple.    Voltage  measurement  require- 
ments:   Varies  between  microvolt  levels  to  tens  of  volts.    Current  measurement  requirements: 

10"      A  to  10"    A.    Environmental  requirements:   Temperature  full  accuracy:    15  C  to  40  C; 
reduced  accuracy:    -10  C  to  +70  C;  humidity  up  to  95%. 

14.  I  would  like  to  give  some  examples  of  electrical  measurement  we  anticipate  in  carrying 
out  medical  device  evaluations: 

1.  Low  level  current  measurement. 

2.  Insulation  resistance  measurement. 

3.  Ground  or  low  resistance  measurements. 

4.  Conducted  and  radiated  EMI  effects  on  instruments. 

a.  amplitude  (current,  voltage); 

b.  frequency  components; 

c.  time  interval  measurements;  and 

d.  energy. 

We  are  currently  evaluating  the  clinical  environment  (temperature,  vibration,  shock,  humidity  and 
electrical)  that  medical  devices  are  subject  to,  and  expect  additional  measurement  problems  to  be 
identified. 

15.  Other  devices  of  interest  include:   (a)    Level  detectors,  voltage  and  current ;   (b)   AC 
voltage,  current,  phase  and  power  measurements  50  to  400  Hz. 

16.  Every  measurement  we  make  now  must  be  reconsidered  in  the  light  of  the  possible 
effect  of  closely-coupled  processing.    Has  the  past  method  compromised  speed  or  accuracy  because 
of  the  choice  of  a  transducer  which  gave  a  more  convenient  numerical  display?   Has  the  accuracy 
been  impaired  by  an  analog  arithmetic  operation?   Has  the  accuracy  been  compromised  by  an 
inability  to  keep  adequate  statistical  data  during  the  experiment?   The  tools  are  now  at  hand  to  over- 
come such  traditional  obstacles  and  open  the  way  to  great  increases  in  the  capability  and  effective- 
ness of  instrumentation. 
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17.  In  the  past,  the  development  of  sensors  for  measuring  various  electrical  and  physical 
quantities  has  concentrated  on  those  devices  which  have  reasonably  linear  outputs  with  respect  to  the 
measured  quantity.    With  the  availability  of  low  cost  microprocessors,  capable  of  linearizing  com- 
plex output  curves,  the  only  real  requirement  is  that  the  sensor  be  precise.    Perhaps  a  whole  new 
generation  of  sensors  could  be  developed  with  characteristics  better  suited  to  today's  technology  if 
the  linearity  constraint  were  removed.    We  often  find  that  a  limiting  factor  in  an  automated  measure- 
ment system  is  not  in' the  instrument  itself,  but  rather  in  the  availability  of  a  compatible  low-cost 
sensor  having  sufficient  precision  for  the  application. 

18.  A  portable  standard  is  required  to  check  phase  angle  of  presently  available  instruments 
in  the  range  of  one  to  thirty  kilohertz  with  an  accuracy  of  +0.  01  degree. 

19.  Improved  phase  angle  measurements:   Accurate  phase  measurements  presently  require 
a  low  distortion  sine  wave  of  constant  frequency.    Often  the  test  signal  has  frequency  instability  and 
is  distorted.    To  make  a  phase  measurement,  a  lengthy  series  of  adjustments  or  several  readings 
and  a  complex  computation  are  needed.    In  addition,  the  phasemeter,  for  high  accuracy,  must  be 
operated  within  a  narrow  ambient  temperature  range. 

20.  Methods  are  needed  for  measuring  BV_,„  _  of  transistors  without  inducing  oscillation. 

A.  1. 2     Signal  Conditioning  and  Data  Conversion 


Attenuators 
Switches,  Relays 
Multiplexers 
Amplifiers 

Sample  &  Hold 

Operational 
Comparators 
Multipliers 

Other  Operators  (log,  etc. ) 
Converters 

Analog-to-Digital 

Digital -to -Analog 
Digital  Voltmeters 


Settling  Times 
Thermal 
Electrical 

Acquisition  Times 

Stabilities 
Long-Term 
Short-Term 

Temperature  Effects 

Noise 

Self- generated 
Immunity  from 


1.  The  signal  conditioning  and  data  conversion  areas  suffer  severely  from  a  lack  of  uniform 
specifications. 

2.  Standardized  specifications  and  calibration  techniques  accepted  by  many  manufacturers 
would  permit  a  customer  to  make  meaningful  comparisons  between  competing  products. 

3.  There  is  a  need  for  improved  accuracy  and  resolution  of  solid  state  programmed  atten- 
uators.   (Example:   0.01  dB  resolution  +1  dB  accuracy). 

4.  Probably  the  most  frequently  mentioned  problem  was  the  need  for  a  solution  to  distrib- 
uting and  switching  high  accuracy  AC  signals  from  audio  frequencies  on  up  through  10  MHz  and  doing 
it  at  high  voltages  as  well.    The  solution  probably  lies  in  a  breakthrough  in  relay  technology. 

5.  Measurement  and  characterization  of  nonlinear  phenomena  as  related  to  precision  equip- 
ment is  a  large  unexplored  territory  -  for  example,  the  measurement  of  dielectric  absorption  and 
its  effect  on  precision  integrators  and  sample -and -hold  amplifiers. 
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6.  A  capability  will  be  needed  to  evaluate  and  characterize  high  performance  operational 
amplifiers  for  such  parameters  as  gain,  stability,  transient  response,  input  offset  currents,  noise 
immunity,  etc. 

7.  Measurements  and  standards  for  high-speed  comparators  need  further  study,  definition 
and  development. 

8.  A  capability  will  be  needed  to  evaluate  electronic  multipliers  as  conversion  devices. 

9.  A  capability  will  be  needed  to  evaluate  analog-to -digital  converters  working  at  high  con- 
version rates  and  accuracies  of  0. 1%  or  better. 

10.  The  current  12-14  bit  devices  are  not  adequate.    Perhaps  a  minimum  resolution  of  14 
bits  with  exponent  bits  to  provide  the  signal  range  would  be  acceptable.    Long  term  accuracy  better 
than  0.  05-0. 1%  does  not  seem  to  be  required. 

11.  What  is  being  done  to  standardize  techniques  for  calibration  of  analog-to-digital  con- 
verters? 

12.  A/D  converters  (including,  but  not  limited  to  DVM's)  are  usually  specified  to  have  a 
certain  accuracy  after  voltage  has  been  applied  for  a  specified  time. 

13.  In  order  to  verify  system  performance  one  must  take  into  account:    1.    The  number  of 
bits  of  accuracy;   2.    Settling  time;  and   3.    Aperture  time.    Conversion  time  may  be  of  interest 
also.    Any  help  in  verification  of  system  performance  would  be  appreciated. 

14.  A  few  specific  ADC  and  DAC  calibration  problems  are  listed  below: 

1.  Adequate  thermal  settling  time  tests. 

2.  Noise  distribution  (often  this  is  important  out  to  +5-sigma). 

3.  Separation  of  noise  and  linearity. 

4.  Practical  temperature  coefficients  testing  (other  than  a  few  points). 

15.  As  a  general  rule,  signals  must  be  limited  to  an  upper  frequency,  about  one-tenth  of  the 
sample  rate  to  achieve  adequate  accuracy.    With  sample  rates  of  moderately  priced  A/D  converters 
currently  in  the  1  megahertz  sampling  rate  range,  dynamic  testing  must  be  limited  to  relatively  low 
frequency  systems.    The  signals  found  in  a  "low  frequency"  system  often  have  significant  compo- 
nents in  the  100,  000  hertz  range.    A  tenfold  increase  in  effective  sampling  rate  is  needed. 

16.  Instrumentation  and  techniques  are  needed  to  measure  single  and  multiphased  time 
intervals  down  to  one  nanosecond  to  an  accuracy  of  0. 1  nanosecond.    (An  example  of  multiphased 
time  interval  is  the  time  delay  between  the  input  and  output  of  a  device. ) 

A.  1. 3     Sources  and  References 

Solid  State  DC 

Voltage  References  Stability 

(all  levels)  Noise 

Self-generated 
Primary  Cells  and  Batteries  Immunity  from 

Transportability 
Reliability 

Environmental  Effects 
Lifetime 
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Generators  and  Measurement  DC  Voltage  and  Current 

Apparatus  AC  Voltage  and  Current 

Sinusoidal,  ramps 
Pulses,  other 
Noise,  Other  Factors 
as  above 
Sampled  Data 
Impulse  Testing 

1.  The  present  unsaturated  cell  voltage  standard  methodology  has  a  number  of  serious 
drawbacks.    Recent  advances  in  band -gap  references  lead  me  to  hope  that  at  some  future  date,  the 
standard  cell  will  be  replaced  with  a  small,  rugged,  temperature-history  independent  device. 

2.  What  is  needed  is  a  primary  DC  standard  which  is  able  to  withstand  moderate  vibrations 
or  shock,  is  self  correcting  for  temperature  variations  over  at  least  a  10  degree  C  range  from  the 
calibration  temperature,  and  returns  to  calibration  accuracy  in  hours  rather  than  days  after  being 
subjected  to  moderate  temperature  extremes  or  shock. 

3.  Production  instruments  are  rapidly  approaching  the  limits  of  NBS  certifiability.    Within 
the  next  5  years,  10  ppm  DC  voltmeters  will  be  as  common  as  100  ppm  units  were  5  years  ago.    The 
demand  is  coming  for  secondary  DC  standards  to  be  accurate  within  1  ppm.    In  essence,  the  need 
exists  to  get  current  secondary  lab  accuracy  out  of  the  standards  lab  and  into  the  shop.    This  implies 
better  automatic  temperature  correction  capability. 

4.  The  standard  voltage  cell  is  a  limiting  factor  in  our  ability  to  provide  calibration  support 
of  digital  voltmeters  and  direct  voltage  standards. 

5.  What  is  being  done  to  replace  standard  cells  with  solid  state  voltage  references?   Is 
economy  to  be  considered  as  a  trade-off? 

6.  Direct  voltage  is  supported  primarily  at  one  volt,  but  the  wide  range  of  DVM's  makes 
voltage  support  at  10  volts,  100  volts,  and  possibly  even  1000  volts,  desirable. 

7.  Fundamental  studies  relating  the  long-term  drift  characteristics  of  Zener  references  and 
differential  amplifiers  to  short-term  measurements  (perhaps  noise  or  high  temperature  perfor- 
mance) would  make  it  possible  to  provide  instrumentation  specifications  that  would  hold  beyond  final 
test  at  the  factory  without  tying  up  expensive  components  in  long  life  tests. 

8.  Battery  life:   Medical  instrument  requirements  such  as  portability  and  electrical  line 
isolation  have  been  most  responsible  for  use  of  battery  power  in  medical  device  development.    As 
the  development  of  cardiac  pacemakers  and  artificial  hearts  progresses,  the  need  arises  to  be  able 
to  assess  the  lifetime  characteristics  of  implantable  power  sources  for  these  devices. 

9.  An  easily  programmed  programmable  calibrator  with  sufficient  flexibility  to  handle  a 
wide  variety  of  functions  and  accuracies  would  be  useful. 

10.  The  characteristics  of  square-wave  and  pulse  sources  are  needed. 

11.  There  is  a  need  to  provide  computer -controlled  signal  sources  that  can  exercise  dyna- 
mically a  large  system  and  confirm  that  it  is  performing  to  specification  over  the  entire  range  of  its 
measurement  capability. 

12.  There  is  a  need  for  improved  high  accuracy  time-varying  amplitude  standards. 
Sample -and -hold  amplifiers  are  used  with  14-and  15-bit  ADC's  and  thus  there  is  a  requirement  for 
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precision  ramps  whose  amplitude  is  known  within  100  ppm  at  any  point  in  time  to  permit  accurate 
tracking  and  aperture  time  measurements. 

13.  Voltage  and  current  measurements  and  their  metrological  support  will  continue  at  high 
level  but  in  the  context  of  rapidly  changing  levels  of  voltage  and  current.  The  question  will  become 
not  how  accurately  can  a  DC  voltage  measurement  be  made  but  how  accurately  can  it  be  made  in  ten 
milliseconds  for  example. 

14.  A  major  problem  is  the  fast,  accurate  measurement  of  voltages  and  currents  (low  level) 
over  a  broad  temperature  range. 

15.  There  is  a  need  for  improved  speed,  range  and  accuracy  in  AC  voltage  measurements. 
(Example:    0.01  dB  resolution  at  1  ms  speed  from  +10  to  -70  dBm,  600  ohms.) 

16.  Ideally,  some  substitute  for  the  slow  responding  thermocouple  is  desired  for  true  RMS 
measurement  of  AC  signals.    The  need  for  increasing  accuracy  into  the  5-10  ppm  area  also  exists. 
We  would  also  like  to  see  higher  voltage  calibration  available  at  the  high  frequencies,  e.g.,  30-50 
volts  at  500  MHz. 

17.  Some  rapid  means  of  calibrating  AC  voltages  to  +0.  01%  from  1  mV  to  1000  volts,  10  Hz 
to  100  kHz  is  required.  Present  techniques  involve  the  use  of  thermal  voltmeters  with  time  expendi- 
tures of  1  hour  or  more. 

18.  Required  is  the  calibration  of  RMS  voltmeters  for  non- sinusoidal  waveforms  (high 
impedance  input),  or  method  of  measurement. 

19.  The  vast  area  of  improving  our  ability  to  test  to  precision  levels  on  semiconductor 
wafer  probing  and  under  computer  control  has  unlimited  areas  for  exploration. 

20.  There  is  a  need  for  current  noise  standards  for  calibration  of  current  noise  meters. 

21.  It  would  be  desirable  if  NBS  could  provide  a  standard  flat  pulse  and/or  certify  aberra- 
tions of  a  pulse  generator  submitted  for  examination. 

22.  Techniques  for  performing  low-level  pulsed  measurements  in  a  high  noise  environment 
are  required.    (Example:    Input  -  offset  voltages  and  currents  for  Op-amps.) 

23.  What  techniques  are  available  for  calibration  of  amplifiers,  filters,  and  transducers  by 
using  step  input  response  or  impulse  techniques? 

24.  If  the  stimulus  is  locally  generated,  an  effectively  high  sample  rate  1/AT  can  be 
achieved  by  repeating  the  stimulus  and  sampling  at  an  interval  equal  to  the  stimulus  sample  interval 
plus  nAT  where   n   is  the  number  of  the  current  repetition.    This  technique  requires  tight  control  of 
repetition  interval  and  AT  as  well  as  a  small  sampling  aperture  or  at  least  high  precision  on  the 
width  of  the  aperture.    A  AT  of  100  nanoseconds  with  an  aperture  accuracy  of  1  nanosecond  is 
needed. 

25.  There  is  a  need  for  simpler,  cheaper,  fully  programmable  oscilloscopes. 

A.  1.4     Systems 
Time  as  a  Parameter  How  accurate,  how  fast 

Noise  as  a  System  Parameter  Generation  and  immunity 
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Systems  Problems  Grounding,  interaction 

interfacing,  switching,  etc. 

Calibration  In  situ  vs.  transporting, 

self-calibration,  optimum 
intervals 

Environment  Temperature,  relative 

humidity 

Software  Program  aided/enabled  measurements 

Program  library 
Applications  programs 
High  level  executives 

1.  In  medium  speed  (10-1000  readings/sec)  measurement  systems,  comprising  typically 
signal  conditioning,  a-d  conversion,  and  some  output  means,  either  visual  or  electrical,  there  are 
many  time  related  performance  parameters  which  are  important  in  automatic  or  semi-automatic 
operation.    These  parameters  are  many  times  subtle,  at  times  difficult  to  measure,  and  typically 
are  inadequately  specified  or  ignored  by  manufacturers.    In  particular,  I  am  referring  to  time- 
dependent  parameters  such  as  settling  time  of  signal  conditioners  as  differentiated  from  a-d  conver- 
sion time  and  total  time  per  digital  output  cycle.    The  settling  time  of  a  signal  conditioner,  or  for 
that  matter  the  a-d  itself,  can  be  a  function  of  its  condition  prior  to  and  during  a  reading;  i.e. , 
overload  recovery,  recovery  from  common  mode  step  or  transient  input,  settling  when  called  from  a 
different  range  or  function,  etc.    All  of  these  time  dependent  parameters  are  a  specification  or  mea- 
surement of  the  response  of  the  system  to  transient  excitation  and  are  directed  at  identifying  the 
degradation  in  performance.    Some  commonly  acceptable  criteria  for  the  description  of  this  degrada- 
tion would  be  most  helpful.    The  techniques  for  the  measurement  of  these  parameters  become 
increasingly  difficult  as  speed  is  increased  and  accuracy  is  improved. 

2.  Required  is  the  measurement  of  settling  time  to  high  amplitude  precision  for  high-speed 
systems  (will  need  0.  001%  in  10      seconds  ultimately);  also,  acquisition  time  measurements. 

3.  What  are  the  limitations  in  performing  AC  parametric  tests  (rise  time,  fall  time,  etc. ) 
on  digital  IC's  on  a  high  volume  basis?  (Example:   What  is  the  shortest  rise  time,  fall  time,  propa- 
gation time  that  can  presently  be  measured  on  a  high  volume  test  system?) 

4.  Investigate  the  economic  realities  of  remote  operation  from  a  central  process  area  ver- 
sus design  of  a  specific  dedicated  system. 

5.  What  is  the  ratio  of  noise  susceptibility  of  an  automatic  calibration  system  to  that  of  a 
typical  laboratory  calibration  setup  ? 

6.  I  would  like  to  mention  noise  sensitivity  in  systems  environments.    Typical  measure- 
ment techniques  and  specifications  for  normal  mode  and  common  mode  rejection  (NMRR,  CMRR)  are 
inadequate  to  describe  performance  in  actual  systems  -  particularly  when  interference  may  be  high- 
level,  broad  spectrum  noise  caused  by  large  analog  signal  switching  or  high  speed  logic  signals. 

7.  Methods  of  eliminating  digital  noise  on  the  output  of  low-level  analog  signals  is  a  neces- 
sity. This  may  require  the  development  of  new  instruments,  modification  of  present  ones  or  devel- 
opment of  improved  filtering  devices. 

8.  The  advent  of  the  mandatory  3-wire  power  cord  with  its  attendant  ground  loop  problems 
will  require  some  consideration,  as  will  guarding  of  instrumentation. 
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9.  What  is  being  done  to  simplify  interfacing  the  instrument  to  be  calibrated  to  the  auto- 
matic calibration  system? 

10.  Evaluate  the  performance  of  these  systems  "in  situ". 

11.  Development  of  reliable  coaxial  switches  is  a  must.    Additional  work  on  the  reliability 
and  repeatability  of  system  switches  (reed,  crossbar,  coaxial,  etc. )  is  required. 

12.  Complex  test  equipment  requires  routine  calibration  "in  situ". 

13.  What  can  be  done  to  resolve  differences  in  test  system  calibration  under  controlled 
temperature  environment  (laboratory)  as  opposed  to  the  widely  varying  temperature  environment  of 
the  field? 

14.  System  Calibration:  The  Air  Force  approach  to  automatic  system  calibration  is  from  a 
system  approach  rather  than  the  calibration  of  individual  modules,  i.e. ,  all  system  parameters  are 
specified  at  the  system  interface. 

15.  A  major  problem  we  (and  I  assume  others)  encounter  with  ATE's  is  to  assure  mainte- 
nance of  calibration  traceability  to  NBS.    The  normal  method  used  is  to  remove  the  component  pieces 
from  the  system,  transport  them  to  the  standards  laboratory,  calibrate  to  specs,  return  to  the  sys- 
tem and  perform  a  functional  test.    We,  as  well  as  many  of  our  instrument  customers,  have  ques- 
tioned this  procedure  in  light  of  the  many  subtle  errors  possible  (the  obvious  ones  include  tempera- 
ture, humidity,  grounds,  etc. ). 

16.  What  is  the  uncertainty  of  measurement  limitations  for  the  automatic  calibration  as 
compared  with  the  typical  laboratory  calibration  setup  ? 

17.  What  factors  should  be  considered  in  establishing  an  acceptable  calibration  interval  for 
the  automatic  calibration  system? 

18.  What  are  acceptable  techniques  for  calibration  of  multiplexed  systems,  i.e.,  frequency 
response  and  linearity? 

19.  Develop  means  to  evaluate  system  immunity  to  temperature,  humidity  and  electrical 
noise. 

20.  Some  hardware  or  software  method  of  insuring  flat  response  of  system  stimuli  would  be 
useful.    Present  methods  use  software  correction  for  cable  and  switch  losses  which  are  determined 
using  thermal  techniques.    A  more  reliable  method  appears  to  be  the  development  of  a  leveling 
device  to  be  used  over  the  frequency  range  from  DC  to  500  MHz. 

21.  The  use  of  software  corrections  to  the  outputs  of  stimuli  should  be  investigated.    This 
will  permit  an  increase  in  system  accuracy  at  the  system  interface  by  incorporating  the  corrections 
found  during  system  calibration.    The  most  feasible  method  of  accomplishing  this  seems  to  be 
through  the  development  of  special  instrument  drivers  made  to  accommodate  the  corrections. 

22.  What  effort  is  being  made  to  implement  a  user  exchange  of  computer  programs  for  the 
automatic  calibration  system? 

23.  Application  Programs:    It  is  quite  apparent  that  the  only  economic  approach  to  applica- 
tion programming  when  a  diverse  load  of  instruments  is  involved  is  to  use  the  generic  approach. 
Continued  development  of  this  technique  is  required  along  with  innovative  methods  of  testing  to  take 
full  advantage  of  the  power  of  automatic  calibration  systems. 
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24.     Development  of  time  shared  real-time  executives  seems  to  be  an  economic  and  practical 
necessity. 

A.  1.5     Other  Input,  Mostly  Having  to  do  with  Calibration 
and  Product  Testing 

1.  The  frequency  of  calibration  and  out -of- service  time  for  calibration  must  be  minimized. 
Various  methods  of  doing  this  include: 

a.  Design  into  the  equipment  a  "calibration  required"  indicator  in  lieu  of  calibrating  at 
fixed  time  intervals. 

b.  Reduce  the  time  items  are  in  transit  to  calibration,  awaiting  calibration,  and  under 
calibration. 

c.  Maximize  the  use  of  designs  which  require  no  calibration. 

d.  Design  in  self-calibration. 

e.  Use  remote  or  on-site  calibration. 

f.  Design  in  interfacing  for  calibration  by  automatic  calibration  systems. 

2.  What  information  is  available  on  calibration  intervals  for  test  instruments?   What  com- 
puter programs  are  available  for  calibration  interval  determination? 

3.  What  techniques  are  available  for  arithmetic  determination  of  long-term  instrument  sta- 
bility by  using  relatively  short-term  data? 

4.  The  Bureau  is  not  permitted  to  endorse  or  imply  endorsement  of  a  specific  manufactured 
product,  but  it  would  be  very  helpful  if  the  Bureau  could  periodically  disseminate  information  on  the 
practical  state-of-the-art  of  various  electrical  and  electronic  measurements  at  various  accuracy 
levels  from  an  objective  viewpoint. 

A.  1.  6     Other  Input,  but  Not  Related  to  the  Work  of  the 
Electricity  Division 
(Relevant  divisions  are  indicated  in  brackets. ) 

1.  Microwave  Power  Measurement:    Production  test  equipment  is  required  to  measure 
power  in  certain  frequency  bands  between  2  and  12  gigahertz  with  an  accuracy  of  0. 1%.    [Electro- 
magnetics Division,  Boulder,   Colorado] 

2.  Microwave  Noise  Measurement:    Production  test  equipment  is  required  to  measure  the 
presence  of  noise  at  a  fixed  frequency  higher  (or  lower)  than  a  carrier  frequency  with  an  output  of  a 
few  watts.    [Electromagnetics  Division,  Boulder,  Colorado] 

3.  We  require  high  vacuum  ionization  gage  calibration  against  an  absolute  standard  in  the 
10"    to  10"    torr  range.     [Heat  Division] 

4.  What  techniques  are  available  for  computer -controlled  calibration  of  accelerometers  ? 
[Mechanics  Division] 

5.  What  techniques  are  available  for  computer-controlled  calibration  of  load  cells? 
[Mechanics  Division] 

6.  What  is  the  state-of-the-art  with  respect  to  calibration  of  velocity  systems?    [Mechan- 
ics, Optical  Physics  Divisions] 

7.  We  have  other  measurement  needs  such  as  good  measurement  of  laser  power,  particu- 
larly peak  power  and  the  design  and  implementation  of  suitable  testing  for  microprocessors  which 
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may  or  may  not  fall  within  the  area  of  discussion  to  be  covered  by  the  Workshop.     [Electromagnetics 
Division,  Boulder,  Colorado] 

8.  The  second  area  which  deserves  greater  attention  is  the  development  of  a  simple,  relia- 
ble and  inexpensive  method  for  evaluating  delivery  of  ultrasonic  energy.    There  are  wide  varieties  of 
both  diagnostic  and  therapeutic  devices  which  employ  ultrasonic  techniques.    Methods  for  evaluating 
the  output  of  sonic  energy  include  force-balance,  water  displacement,  calorimetry  and  calibrated 
transducers.    These  techniques  are  in  many  instances  cumbersome,  time-consuming  and  compli- 
cated.   To  obviate  some  of  these  drawbacks,  research  and  development  into  better  and  more  accu- 
rate calibrated  transducer  techniques  could  be  considered.    [Electromagnetics  Division,  Boulder, 
Colorado;  Mechanics  Division] 

9.  What  instrumentation  is  available  for  dynamic  calibration  of  accelerometers  and  load 
cells  at  high  levels?    [Mechanics  Division] 

10.  What  is  being  done  to  standardize  measurement  of  spring  rate  and  damping  of  elasto- 
mers as  used  in  dummies?   [Mechanics  Division] 

A.  2     Notes  on  Opening  Plenary  Session 

These  notes  were  prepared  by  the  moderator  and  are  based  on  the  discussion  at  the  opening 
plenary  session.    The  purpose  of  the  session  was  to  introduce  the  attendees  to  the  National  Bureau  of 
Standards  and  the  Electricity  Division  in  particular,  to  ensure  that  everyone  understood  the  goals  and 
modus  operandi  of  the  Workshop,  and  finally,  to  begin  the  discussion  of  the  four  main  topical  areas. 
The  notes  deal  primarily  with  the  latter. 

Although  many  specific  needs  were  touched  upon  during  the  opening  session,  some  broad 
major  problem  areas  were  also  identified. 

Interpretation  of  device  specifications  was  one  of  the  most  important.    It  was  felt  that  often, 
NBS  documents  are  written  for  the  user  who  is  already  highly  knowledgeable  in  the  specific  area, 
while  the  people  really  needing  help  are  those  just  using  the  device,  module,  instrument  or  technique 
for  the  first  time.    Thus,  there  would  seem  to  be  a  great  need  in  the  field  for  statements  on  how 
specification  facts  are  derived,  and  on  measurement  limitations.    To  this  end,  the  possible  function 
of  the  Division  in  a  tutorial  role  was  suggested.    The  Division  could,  for  example,  provide  recom- 
mended calibration  procedures  -  not  tied  to  specific  instruments,  but  rather  to  specific  types  of 
measurements. 

A  second  major  theme  was  time  as  a  parameter  in  electrical  measurements.    A  typical 
example,  and  a  kind  of  measurement  presently  more  or  less  ignored,  is  recovery  from  high  voltages 
applied  to  low  voltage  scales  -  1000  V  on  the  100  mV  scale  of  a  digital  voltmeter,  for  instance.    The 
question  of  how  long  a  wait  is  necessary  to  restore  rated  accuracy  is  seldom  addressed,  and  is  be- 
coming increasingly  important  in  systems  applications. 

A.  2. 1     Components  and  Parameter  Measurements 

Binary- oriented  standards  and  methodology  would  be  more  consistent  with  many  systems 
applications. 

In  the  past,  we  were  designing  instrumentation  for  the  human  being.    Now,  with  computers, 
we  can  readily  correct  for  significant  non-linearities,  hence  we  have  more  options. 

What  is  of  increasing  importance  is  stability  and  reliability  -  advanced  prediction  of  perfor- 
mance. Work  is  needed  on  the  basic  physics  of  devices  for  purposes  of  stability  and  failure  predic- 
tion. 
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Also  important  is  the  prediction  of  the  value  standards  will  have  in  the  future  via  extrapola- 
tion of  measurements  made  today. 

Increased  accuracy  is  needed  in  AC/DC  conversion  -  15  to  30  ppm  accuracy  out  to  100  kHz  is 
required. 

Systems  must -eventually  give  automatic  indication  of  out-of-calibration  conditions. 

With  systems,  the  important  place  for  meeting  specifications  is  at  the  customer  interface, 
e.g. ,  where  the  instrument  under  test  is  connected  to  the  system. 

A. 2. 2     Signal  Conditioning  and  Data  Conversion 

Dynamic  accuracy  is  the  major  area  of  difficulty.    We  need  to  understand  the  role  of  settling 
times  in  transient  measurements.    Perhaps  frequency  domain,  via  crosstalk  measurements  on 
DAC's  or  ADC's  used  for  speech,  is  the  way  to  go. 

In  sampled  data  measurements,  inaccuracy  is  seen  as  noise.    What  is  needed  is  a  charac- 
terization of  the  noise  -  dynamic  features  of  the  error  -  such  as  peak  amplitudes  and  spectral  com- 
position. 

It  is  important  to  distinguish  between  correlated  and  uncorrelated  noise. 

In  the  operational  dynamics  of  specifications,  once  you  have  defined  how  to  measure  the 
noise,  you  have  then  specified  it. 

Failure  mode  investigation  is  a  significant  area  for  Division  effort. 

Ground  currents  in  systems  can  cause  strange  phenomena  in  measurements. 

Of  great  potential  benefit  are  Bureau-certifiable  standards  for  field  use  at  higher  accuracies 
than  are  presently  available. 

hi  the  field,  instruments  and  systems  have  pickup  problems  from  impulse  sources.    Also 
needed  is  equipment  to  perform  IEEE  Surge  Withstand  test. 

The  sampled  data  approach  is  really  a  holdover  from  manual  techniques.    It  should  be  possi- 
ble to  just  take  a  few  measurements,  on  the  fly,  then  process  them  to  predict  the  final  value.    The 
difficulty  arises  in  relating  results  to  what  we  already  know.    A  particular  example  might  be 
impulse  response  testing. 

A.  2. 3     Sources  and  References 

Sampling  measurements  occasionally  give  erroneous  data,  i.e. ,  data  outliers.    There  is  no 
easy  means  for  recording  such  transients. 

The  3-sigma  limit  is  often  called  into  play  to  take  care  of  the  outlier  problem  in  specifica- 
tions. 

Lifetimes  of  implantable  energy  sources  must  be  evaluated  in  advance. 

A  further  problem  is  to  determine  when  Ni Cad's  or  other  batteries  cannot  be  recharged 


again. 


A  certified  flat  pulse  is  crucial  to  settling  time  measurements. 
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Cables  and  terminations  are  often  crucial  system -limiting  elements. 

A.  2. 4     Systems 

It  would  be  helpful  if  the  Electricity  Division  were  to  disseminate  information  about  its  on- 
going work. 

Software  is  the  biggest  single  system  problem  area  -  it  would  be  helpful  if  the  Bureau  could 
make  recommendations  on  preferred  approaches,  etc. 

Calibration  of  sensors  and  transducers  is  an  important  area. 

Adding  protective  devices  to  a  design  often  changes  the  calibration,  yet  this  is  not  widely 
known  and  its  implications  are  not  fully  recognized. 

It  would  be  extremely  useful  to  have  a  specification  on  the  standard  AC  power  line  for  all 
types  of  performance  including  spikes  due  to  lightning,  etc. 

A.  3     Working  Sub -Group  Reports 

The  following  four  working  sub-group  reports  were  prepared  by  the  cognizant  NBS  Recorder 
with  the  assistance  of  the  appropriate  NBS  Participant  and  some  additional  input  from  the  working 
sub-group  Chairman.    Final  editing  was  the  responsibility  of  the  moderator.    The  reports  cover  all 
of  the  working-group  deliberations  carried  out  during  the  course  of  the  Workshop.    It  should  be 
noted  that  while  the  sub-group  participants  changed  during  the  two  days,  the  Recorder,  NBS  Partici- 
pant, and  Chairmen  remained  the  same,  thereby  providing  continuity. 

A.  3. 1     Components  and  Parameter  Measurements 
(E.  Boeckmann,  Chairman;  M.  Oldham,  NBS  Recorder;  W.  Eicke,  NBS  Participant) 

A .  3 . 1 . 1     Introduction 

The  need  for  dynamic  rather  than  static  measurements  of  components  was  voiced  by  nearly 
all  of  the  members  of  this  sub-group.    Rapid  DC  response  was  considered  more  important  than  the 
conventional  frequency  response  to  sinusoidal  signals.    Requested  frequencies  of  these  "pulse"  mea- 
surements ranged  from  10    Hz  to  10    Hz  (50%  duty  cycle);  and  1    V  to  10    V   energizing  voltages, 
variable  pulse  to  pulse,  would  be  desirable. 

A.  3. 1. 2     Capacitors 

The  sampling  rate  and  accuracy  of  sample -and -hold  amplifiers  are  functions  of  their  hold 
capacitors'  pulse-frequency  response.    In  order  to  improve  these  and  other  devices,  more  informa- 
tion on  capacitor  parameters,  specifically  dielectric  absorption,  must  be  made  available.    It  was 
generally  felt  that  NBS  could  be  of  great  assistance  to  the  industry  in  the  following  ways: 

1.  NBS  should  set  measurement  guidelines  and  specify  test  conditions  for  these  parameters. 
Pulse -response  is  preferred  to  static  tests.    NBS  should  also  establish  a  calibration 
service  in  this  area  with  a  possible  MAP. 

2.  Dielectric  absorption  measurements  on  capacitors  from  10  pF  to  1    /uF  with  an  accuracy 
of  at  least  1000  ppm,  and  possibly  as  high  as  10  ppm,  are  desired. 

3.  Standards  of  dielectric  absorption  were  discussed  briefly,  with  polystyrene  and  non- 
solid  dielectrics  being  of  prime  interest.    The  guarded  air  capacitor  was  recommended 
as  a  possible  standard. 
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4.      It  was  generally  felt  that  NBS  could  also  assist  the  industry  by  doing  more  basic  dielec- 
trics research. 

A.  3. 1.3     Resistors 

Sixteen-bit  processors  which  resolve  1  part  in  65000,  and  fast  laser  trimmers  which  trim 
many  resistors  a  second,  are  two  examples  where  high  accuracy  resistors  are  used  in  high  speed 
measurements.  It  was  generally  agreed  that  a  "pulse-response"  resistor  calibration  service  was 
needed  to  cover  the  following: 

1.  Resistors  used  in  the  ladders  and  other  types  of  networks  used  in  DAC's  and  ADC's. 
Generally,  the  range  is  from  10z  Q  to  10    Q.    Present  requirements  are: 

a.  0. 1%  pulse  response  to  rep  rates  of  10    Hz 

b.  1%  pulse  response  to  rep  rates  of  10y  Hz 

2.  It  has  been  requested  that  NBS  offer  design  considerations  for  high  speed  resistors; 
improved  theory  is  needed.    Requests  were  made  for  improved  film  resistor  design. 
Ten  ppm  long-term  stability  is  the  goal. 

3.  Binary  values  are  being  used  more  and  more  in  modern  processors.    The  need  for  a 
calibration  service  for  other  than  base  10  resistors  is  increasing. 

4.  Temperature  coefficient  measurements  have  been  a  particular  problem  to  some.    It  was 
not  clear  why  there  is  so  much  confusion  measuring  this  parameter  but  at  the  mention  of 
a  special  calibration  service  for  resistor  temperature  coefficient,  several  resistor 
manufacturers  indicated  interest.    There  was  also  interest  in  a  MAP. 

5.  In  16-bit  processors,  resistor  ladder  and  other  network  ratios  must  be  known  quite  well. 
There  is  a  present  need  for  static  DC  ratio  measurements  to  an  uncertainty  of  1  ppm. 
Some  manufacturers  of  these  networks  calibrate  them  statically.    They  are  then  used  in 
high  resolution  processors  under  a  variety  of  dynamic  conditions,  apparently  on  the 
assumption  that  thin  films  have  good  frequency  characteristics.    The  need  for  network 
ratio  pulse -response  was  only  touched  upon  and  no  figures  were  given.    It  was  generally 
agreed  that  a  DC  or  low  frequency  (LF)  AC-to-pulse  transfer  technique  would  be  useful, 
so  that  a  one  time  difference  could  be  measured  and  a  DC  or  LF  technique  could  be  used 
thereafter  for  recalibration. 

6.  The  measurement  of  resistance  from  10    Q,  to  10      Q,  was  of  general  interest.    There 
was  agreement  that  NBS  should  set  measurement  guidelines  with  emphasis  on  the  param- 
eters which  need  specification.    Commercial  units  lack  stability  and  improved  designs 
are  needed.    It  was  mentioned  that  NRC  offers  a  calibration  service  at  10°  Q  with  a 
slightly  smaller  uncertainty  than  NBS.    It  was  pointed  out,  however,  that  the  present  NBS 
uncertainty  from  10    fi  to  10      Cl  is  less  than  35  ppm  and  less  than  0. 1%  from  10      Q  to 
10      Q .    This  accuracy  is  not  offered  because  of  the  instability  of  the  resistors  sent  in 
for  test. 

A. 3. 1.4     Phase  Angle  Measurements 

Currently  marketed  phase  meters  claim  the  ability  to  measure  the  phase  angle  between  two 
voltages  to  0.01  degree.   A  calibration  service  is  not  available  at  NBS  and  apparently  no  traceable 
measurements  are  available  in  the  industry.    The  project  on  phase  standards  currently  being  under- 
taken in  the  Electricity  Division  was  discussed  briefly. 
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Phase  angle  of  component  impedances  was  discussed  more  thoroughly.    Some  resistor  - 
capacitor  measurements  presently  done  at  NBS  offer  or  could  offer  a  phase  angle  determination  to 
varying  degrees  of  accuracy.    Although  these  measurements  are  made  with  sinusoidal  signals,  it  is 
felt  that  the  phase  angle  may  be  a  valuable  tool  in  determining  pulse -response. 

A.  3. 1.5     Noise 

Noise  measurements  and  noise  standards  were  discussed  briefly,  with  emphasis  on  NBS 
guidelines  and  specifications.    People  concerned  with  noise  were  most  interested  in  methods  of  mea- 
surement that  could  possibly  be  supplied  by  NBS.    Noise  standards  from  10  kHz  to  500  kHz  were 
needed  not  only  for  noise  measurement  but  to  determine  frequency  response.    White  or  gaussian 
noise  was  discussed,  with  Zener  diodes  mentioned  as  possible  standards.    Uses  of  standard  noise 
generators  included  component  (capacitors  and  resistors)  and  product  (Op-amps,  hearing  aids) 
evaluation.    Thin  and  thick  film  resistor  noise  was  also  of  particular  interest;  more  theory  is 
needed.    It  is  felt  that  current  noise  (apparently  due  to  film  resistors  and  semiconductors)  may  be 
the  limiting  factor  in  ADC  design. 

A.  3. 1.6     Other  Topics 

1.  The  need  for  more  accurate  measurement  of  distorted  power  was  voiced  but  not  elabo- 
rated upon. 

2.  A  request  for  an  NBS  calibration  service  for  inductors  from  100  piH  to  10  H,  was  made. 
Frequencies  from  200  Hz  to  2000  Hz  at  accuracies  from  10  to  50-ppm  are  desired. 

3.  It  was  generally  felt  that  NBS  should  conduct  a  "pulse"  study  before  getting  too  deeply 
into  pulse-response  measurements. 

4.  Transducer  interfacing  has  been  a  constant  problem  to  many  people.    Transducers  may 
respond  differently  to  different  readouts.    Greater  interaction  between  the  transducer 
testing  laboratories  (at  NBS)  and  the  customer  is  indicated. 

5.  Methods  of  short-term  evaluation  of  long-term  performance  are  needed  by  all  segments 
of  the  industry  where  lifetime  is  a  factor.    Resistor  and  network  stability  was  of  prime 
concern  but  the  evaluation  of  medical  implant  batteries  was  of  interest  to  FDA. 

6.  Of  further  interest  to  FDA  was  calibration  support  for  medical  monitors  and  analyzers. 
There  is  presently  no  control  on  these  devices  once  they  are  purchased  and  calibration 
services  are  hard  to  find. 

A.  3. 2     Signal  Conditioning  and  Data  Conversion 
(D.  Ludwig,  Chairman;  T.  Souders,  NBS  Recorder;  H.  Schoenwetter,  NBS  Participant) 

A.  3 . 2 . 1     Introduction 

The  Sub-Group  opened  with  a  discussion  of  the  proprietary  aspect  of  measurement  problems, 
and  a  suggestion  that  not  all  members  might  want  to  discuss  problems  among  each  other.    As  a 
result,  NBS  might  have  to  re-invent  measurement  techniques  withheld  by  manufacturers  that  believe 
the  ability  to  make  better  measurements  gives  them  an  edge  in  building  better  equipment.    Others 
felt  that  good  test  methods  should  benefit  all,  and  hence  should  not  be  proprietary.    In  an  anecdote 
relating  past  experience  with  NBS  on  inductive  voltage  dividers,  one  participant  pointed  out  that  in 
the  long  run  manufacturers  will  benefit  from  educating  NBS  about  their  measurement  techniques.    In 
the  example,  NBS  eventually  "took  the  ball",  and  was  able  to  offer  technical  assistance  which  the 
manufacturers  could  not  afford  on  their  own.    NBS's  knowledge  and  approval  helped  the  market  as 
well. 
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What  sort  of  role  should  NBS  play?   Since  parameters  must  be  well  defined  before  meaningful 
measurements  can  be  made,  NBS  could  start  by  characterizing  devices  realistically  (as  opposed  to 
some  Mil-Specs),  considering  input  from  both  manufacturers  and  users.    Considerable  concern  was 
expressed  by  manufacturers  that  this  characterization  be  realistic,  and  not  tie  their  hands.    Next, 
NBS  could  develop  standard  testing  procedures  suitable  for  the  user,  who  will  benefit  most  from  the 
measurements.    Such  standard  procedures  would  be  taken  as  a  yardstick  for  determining  the  correct 
values,  with  any  other  technique  giving  comparable  results  being  acceptable.    In  this  way,  manufac- 
turers need  not  be  limited  to  NBS's  recommended  techniques  for  production  testing  if  other  tech- 
niques give  acceptable  results. 

It  was  pointed  out  that  in  many  instances,  linearity  is  much  more  important  than  absolute 
accuracy,  the  latter  being  achieved  by  the  user  with  external  or  built-in  "twiddlers".    Hence,  NBS 
should  probably  concentrate  its  efforts  on  linearity  rather  than  accuracy. 

A. 3. 2. 2     Settling  Time 

Settling  time  is  a  measurement  parameter  common  to  most  analog  circuits,  e.g. ,  amplifi- 
ers, DAC's,  ADC's,  sample-and -holds,  as  well  as  R,   L,  C  components  and  integrated  components. 
Four  years  ago,  no  specifications  were  available  on  this  parameter,  and  even  now  no  products  are 
available  to  measure  it.    An  accuracy  of  0.01%  in  20  ns  should  be  strived  for,  but  50  or  100  ns 
would  still  be  very  useful.    Because  of  thermal  drifts,  etc.,  of  the  settled  value,  a  definition  is 
needed  to  determine  when  the  final  reference  value  is  taken.    The  measurement  technique  must  be 
made  suitable  for  use  with  a  reference  input  pulse  (as  from  a  well  characterized  flat-top  generator) 
for  amplifiers,  ADC's,  etc.,  as  well  as  for  use  without  any  reference  signal,  as  in  a  DAC.    A  ref- 
erence source  should  be  a  flat -top  generator  adjustable  to  known,  precision  levels  (settling  time  is 
not  linear  and  should  be  measured  over  a  wide  dynamic  range),  having  both  polarities  available, 
having  well  controlled  rise  time  with  low  overshoot,  and  a  top  holding  flat  for  at  least  1  ms. 

NBS  should  consider  the  possibility  of  a  transportable  standard  of  settling  time.    It  was  sug- 
gested that  settling  time  and  accuracy  be  kept  independent,  i.e.,  the  accuracy  can  be  tested  after  the 
device  is  well  settled.    Dynamic  accuracy  testing  however,  may  be  helpful  if  the  final  value  is  drift- 
ing. 

Although  great  difficulties  are  anticipated,  the  measurement  of  settling  time  on  a  wafer - 
probe  basis  would  be  helpful. 

It  was  pointed  out  that  even  though  some  devices  do  not  really  settle  during  the  observed  time 
interval,  the  lack  of  settling  is  constant,  and  can  be  compensated  for. 

A  large  disparity  in  measurement  ability  exists  between  manufacturer  and  user,  hence  the 
user  will  benefit  most  by  NBS's  work  in  this  field. 

A.  3. 2. 3     Characterization  of  Data  Conversion 

Characterization  of  data  conversion  and  its  dynamic  test  conditions  was  a  commonly  men- 
tioned problem.    The  dynamic  input  impedance  of  ADC's  and  DAC's,  for  example,  needs  specifica- 
tion.   How  many  different  operating  points  should  be  tested?   How  are  noise  and  uncertainties  in  ADC 
transition  points  (e.g.,  due  to  past  history)  to  be  specified?   How  are  the  make -before -break  dynam- 
ics of  multiplexers  characterized?  (Low-level  sources  may  have  high-level  voltages  momentarily 
applied  to  them  by  make-before-break  action. ) 

Dynamic  response  specifications  and  test  methods  are  needed  for  voltage-to-frequency  con- 
verters.   Temperature  hysteresis  is  a  problem  with  them. 
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A.  3. 2. 4     Noise 

Non-gaussian  noise  is  prominent  in  semiconductor  circuits.    Better  understanding  of  the  dis- 
tribution and  effect  of  popcorn  noise,  for  example,  is  needed,  along  with  a  more  meaningful  charac- 
terization, since  no  conclusions  as  to  peak-to-peak  noise  can  be  drawn  from  RMS  measurements. 
Correlated  noise,  e.g. ,  chopper  noise,  is  another  common  non-gaussian  noise  to  be  considered. 
Him  resistor  noise  is  still  another  area  needing  further  study. 

A.  3.2.5     Sample -and -Hold  Measurement  Problems 

Again,  definitions  of  parameters  such  as  aperture  time  and  acquisition  time  are  needed  as 
well  as  methods  for  measuring  them.    Means  are  needed  for  both  characterizing  and  studying  mem- 
ory effects  on  a  components  level,  e.g. ,  studies  of  soakage  or  dielectric  absorption  of  capacitors 
over  short  time  periods.    NBS  could  serve  well  in  this  area. 

Development  of  special  test  generators,  such  as  precision  ramps  (up-down  with  corners), 
would  provide  a  valuable  measurement  tool.    Measurement  of  acquisition  time  should  strive  for 
0.01%  in  20  ns,  and  aperture  uncertainty  should  be  known  to  100  ps  or  better. 

A. 3. 2. 6     ADC's  and  DAC's 

A  precision  programmable  power  supply  or  DAC,  having  10  ppm  or  better  accuracy,  with 
floating  (+1  V)  output  would  be  helpful  for  testing  ADG's  and  DAC's. 

Multiplying  DAC s  are  important  large-volume  market  items,  and  present  characterization 
problems.    Could  NBS  develop  a  standard  recommended  test  procedure  which  a  user  could  employ? 

A.  3.2.7     Comparators 

Comparator  noise  is  an  important  consideration  in  determining  the  real  accuracy  of  ADC's. 
Comparator  response  time  needs  to  be  specified  for  small  as  well  as  large  error  signals. 

A.  3. 2.  8     Nonlinear  Modules 

The  problem  with  nonlinear  modules,  e.g.,  XY,  XY/Z,  log  X,  is  predominantly  one  of  defi- 
nition and  characterization.    Accuracies  are  usually  modest  (^1%)  and  rather  easy  to  measure. 
Interest  was  expressed  in  RMS  measurements  for  nonsinusoidal  waveforms  with  crest  factors  in  the 
upper  range  of  10-12,  full  scale. 

A.  3. 2. 9     Long  Term  Performance,  Temperature  Effects 
and  Component  Studies 

Correlating  long-term  stability  with  short  term  measurements,  such  as  cycling  and  burn-ins, 
is  a  very  important  goal.    No  good  means  are  at  hand  for  studying  the  problem,  and  manufacturers 
cannot  afford  the  expense  to  make  the  necessary  studies.    It  was  suggested  that  NBS  is  perhaps  the 
only  reliable  group  that  could  conceivably  afford  to  make  such  studies.    As  a  start,  it  was  proposed 
that  NBS  study  component  stability  (film  resistors,  capacitors,  semiconducting  devices)  with  a  view 
towards  evaluating  long  term  stability  and  temperature  effects.    The  stability  of  resistor  ratios  was 
emphasized  for  this  study.    The  semiconductor  studies  of  the  Electronic  Technology  Division  were 
used  as  an  example  of  the  type  of  work  needed. 

Characterization  of  temperature  effects  was  considered  a  big  problem,  and  this  can  become 
even  more  confusing  in  monolithic  circuits  when  the  thermal  time  constants  become  so  small  that 
they  look  like  other  time  problems. 
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A.  3. 2. 10     Financial  Impact 

It  was  estimated  that  better  specification  and  measurements  might  double  the  gross  volume  of 
the  signal  conditioning  and  data  conversion  market,  which  presently  amounts  to  about  $50  million 
annually  for  DAC's  and  ADC's  alone.    One  example  was  given  of  a  $200,  000  loss  due  to  mis- 
specification  of  an  order.    This  would  have  been  prevented  had  standard  specifications  or  characteri- 
zation been  available. 

A.  3. 3     Sources  and  References 
(D.  Lawrence,  Chairman;  B.  Field,  NBS  Recorder;  R.  Turgel,  NBS  Participant) 

A.  3. 3.1     DC  Standards 

In  the  discussion  of  DC  standards,  the  shortcomings  of  saturated  standard  cells  were  empha- 
sized.   Cells  are  used  today  as  secondary  standards  because  of  their  stability  and  low  noise;  how- 
ever, they  are  unsuitable  for  field  use  or  production-line  testing  because  they  are  sensitive  to  the 
environment,  specifically  temperature.    Temperature  hysteresis,  requiring  recovery  time  of  24 
hours  or  longer,  or  possibly  even  permanent  change,  is  one  of  the  problems.    Recalibration  of  cells 
that  are  physically  transported  takes  at  least  two  weeks,  even  under  favorable  conditions.    The  cells 
provide  only  one  volt  output  with  no  power  (i.e.,   "zero"  current).    This  means  that  1  ppm  (1  /zV)  is 
within  the  noise  range  of  most  amplifiers.    Transfer  type  measurements  are  required  for  other 
voltage  and  impedance  levels.    In  the  standard  cell,  low  noise  is  combined  with  high  internal  imped- 
ance, but  selected  solid  state  devices  with  relatively  low  output  impedance  can  be  produced  with 
noise  as  low  as  1  ppm. 

An  alternative  to  a  saturated  cell  is  required,  but  it  need  not  necessarily  be  a  solid  state 
device.    For  example,  a  mercury  battery,  periodically  checked  by  an  ATE,  could  be  used  as  a  short 
term  reference,  because  it  has  a  predictable  drift  rate.    As  another  example,  the  "volt"  could  be 
piped  around  via  a  cable  using  a  constant  current  source  to  eliminate  thermal  emf's.    However,  this 
would  cost  more  than  individual  Zener  diodes,  which  can  have  a  stability  of  1  to  5  ppm  over  the  tem- 
perature range  of  10  to  50  C,  and  are  more  portable.    Present  solid  state  devices  have  temperature 
coefficients  of  2  ppm/  C  over  a  15  to  50  C  temperature  range.    The  coefficient  is  worse  over  the 
military  temperature  range,  but  environmental  control  could  be  used  for  compensation.    A  Zener 
diode  can  compete  in  accuracy  with  an  unsaturated  cell,  if  the  current  through  the  diode  is  held  con- 
stant.   The  noise  of  the  diode  depends  on  the  current  used. 

For  many  applications,  what  is  wanted  is  a  device  that  provides  a  wide  range  of  voltages  and 
currents.    These  secondary  standards  should  approach  the  accuracy  of  today's  primary  standards. 
Ten  ppm  digital  voltmeters  are  no  longer  out  of  the  question;  at  least  one  manufacturer  is  working  in 
this  area.    A  5  ppm  standard  is  inadequate  for  testing  these  instruments.    If  ATE  will  be  used  to 
calibrate  the  DVM's,  then  we  need  standards  capable  of  1  ppm  accuracy  with  low  noise.    These  are 
desirable  to  eliminate  the  need  for  integrating  measurements  over  a  long  period  to  reduce  the  effect 
of  noise  in  the  standards.    Long  integration  times  decrease  the  throughput  of  the  ATE. 

In  summary,  we  want  a  new  standard,  an  alternative  to  the  standard  cell.    It  should  have 
greater  environmental  tolerance,  1  to  5  ppm  stability  over  10  to  50  C,  and  when  held  at  a  constant 
temperature,  have  stability  of  1  ppm.    The  noise  should  be  less  than  1  ppm.    It  should  have  10  V  and 
1  volt  (1.  018  . . . )  outputs,  and  be  able  to  supply  current  up  to  10  mA.    Additionally,  it  should  show 
no  temperature  hysteresis  and  have  a  1-hour  or  less  warm-up  time.    It  was  concluded  that  there 
was  an  immediate  need  for  2  to  5  ppm  Zener  standards  and  a  suitable  calibration  service  at  the  one 
and  ten  volt  levels.    The  representative  of  one  module  manufacturer  in  particular,  felt  that  a  MAP- 
type  program  for  Zeners  should  be  provided,  at  least  at  the  10  ppm  level  if  not  better.    He  stated 
that  he  would  be  in  a  position  to  use  this  service  in  1  to  5  years. 
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Ideally,  this  secondary  standard  should  be  constant,  or  at  least  behave  predictably.    If  device 
behavior  can  be  predicted,  then  corrections  can  be  made,  for  example,  by  a  microprocessor 
included  in  the  unit.    Corrections  could  be  for  temperature  effects  and/or  drift.    It  was  suggested 
that  an  investigation  be  made  of  possible  correlations  between  short-term  Zener  parameters  and  long- 
term  stability.    An  effective  method  capable  of  predicting  the  long-term  performance  would  have 
great  economic  impact.    This  would  enable  manufacturers  to  spend  less  time  testing  Zeners,  thus 
producing  reliable  Zener  references  (and  DVM's)  at  lower  cost.    NBS  could  possibly  evaluate  the 
long-term  performance  of  Zeners  produced  by  industry.    The  representative  of  one  instrument 
manufacturer  mentioned  that  at  present,  in  DAC's,  references  are  adequate  and  frequency  compen- 
sation of  the  voltage  divider  is  the  critical  problem. 

A.  3.3.2     Band-Gap  Reference 

An  alternative  to  the  Zener  diode,  a  band-gap  reference,  was  suggested.    The  possible  appli- 
cation of  this  device  as  a  voltage  reference  was  reported  by  K.  E.  Kuijk  in  the  June  1973  issue  of  the 
IEEE  journal  of  Solid  State  Circuits.    This  device  is  temperature  dependent,  however,  the  environ- 
ment could  be  controlled.    The  advantages  are  that  this  device  is  based  on  an  intrinsic  property  of 
silicon,  and  can  be  operated  from  a  five  volt  supply  voltage  and  thus  can  be  included  in  logic  devices 
without  the  requirement  of  an  additional  power  supply.    One  participant  had  the  impression  that  Mr. 
Kuijk  was  making  devices  with  a  stability  as  good  as  10  ppm  a  year. 

It  was  pointed  out  that  as  Zener  diode  stability  improves,  the  needed  accuracy  of  the  stan- 
dards required  to  calibrate  them  would  increase.    Thus  work  should  continue  in  making  the  Joseph- 
son  effect  voltage  standard  easier  to  use. 

A.  3.3.3     Noise  Sources  and  Standards 

A  number  of  questions  were  raised,  including:    Is  there  a  standard  for  noise  specification? 
What  type  of  device  should  be  used  to  measure  noise,  and  how  do  people  measure  how  much  noise  is 
in  a  circuit?   What  happens  to  an  AC  meter  when  a  noise  source  is  put  on  the  input?   Should  noise  be 
measured  with  an  RMS  meter?  With  a  filter?  We  need  a  standardized  procedure  to  measure  white 
noise  from  10  Hz  to  30  MHz.    An  RMS  measurement  is  theoretically  correct.    Possibly  AC  devices 
could  be  calibrated  by  noise  sources  instead  of  sine  waves.    Perhaps  a  one  point  measurement  could 
be  made  using  a  calibrated  noise  source  instead  of  a  multipoint  frequency  test.    It  became  evident  in 
the  discussion  that  there  is  little  or  no  agreement  on  how  noise  should  be  specified  and  measured. 

A.  3. 3. 4     AC-DC  Transfer 

The  problem  of  AC-DC  transfer  is  directly  connected  with  the  problem  of  developing  and 
checking  precision  AC  sources.    AC  sources  must  be  capable  of  supplying  the  currents  necessary 
for  the  thermal  converters  with  which  they  are  tested.    At  high  voltages  the  power  required  is  not 
negligible.    The  impedances  presently  are  determined  by  frequency  considerations,  but  a  wide  range 
instrument  would  reduce  the  supply  requirements.    Most  thermocouples  have  to  be  used  between  50% 
and  100%  of  full  scale.    This  necessitates  having  many  devices  on  hand.    An  instrument  that  could  be 
used  from  10%  to  100%  of  full  scale  would  be  more  useful.    One  participant  commented  that  promptly 
publishing  work  done  at  NBS  on  AC-DC  transfer  techniques  would  help  bring  the  accuracy  obtainable 
in  industry  more  in  line  with  the  state-of-the-art.    Accuracies  of  calibrated  AC-DC  converters  at 
audio  frequencies  were  generally  considered  to  be  sufficient  for  the  present,  however,  10  ppm  was 
desirable  in  the  future  so  that  30  to  50  ppm  could  be  obtained  in  the  field.    The  major  restriction 
v/as  at  high  frequencies  (1  MHz  to  100  MHz).    Very  high  frequency  satellite  communication  signals 
need  to  be  measured  at  70  MHz  to  100  MHz  to  better  accuracy  than  is  presently  available.     For 
instance,  a  13-MHz  generator  has  been  developed  with  a  resolution  of  0.  01  dB  and  cannot  be  cali- 
brated to  that  accuracy  because  of  a  lack  of  a  suitable  NBS  calibration  service.    On  the  other  hand, 
it  was  commented  that  accuracy  requirements  for  AC-DC  converters  should  be  examined  as  to  user 
needs,  and  not  product  requirements. 
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A  major  problem  with  thermocouple  devices  is  that  they  are  slow.    A  faster  responding 
device  is  needed.    Digital  sampling  is  a  possibility  and  will  allow  a  certain  amount  of  harmonic  con- 
tent without  degrading  the  accuracy.    Even  for  a  small  number  of  samples  a  good  RMS  reading  can 
be  calculated.    We  need  an  RMS -responding  device  with  time  constant  a  factor  of  10  lower  than  pres- 
ent thermocouples,  that  is  the  ability  to  make  10  measurements  a  second  at  frequencies  up  to  1  MHz, 
with  a  minimum  accuracy  of  0. 1%.    The  system  would  then  be  compatible  with  an  ATE,  and  time- 
consuming  spectral  purity  measurements  (needed  with  average  response  readings)  could  be  elimi- 
nated.   This  would  result  in  increased  throughput.    Reduction  of  the  DC  reversal  error  (or  alter- 
nately, the  ability  to  predict  the  error)  will  also  reduce  the  time  required  to  make  the  measurement. 

In  some  cases  the  RMS  response  to  nonsinusoidal  AC  quantities  depends  on  the  crest  factor. 
A  generator  that  is  calibrated  in  crest  factor  could  be  of  value  for  such  AC  calibration. 

A.  3. 3.  5     Battery  Life /Capacity 

There  exists  a  need  to  determine  the  remaining  charge  of  a  battery  without  discharging  the 
battery.    Batteries  used  to  power  standard  cell  enclosures  during  shipment  and  batteries  implanted 
in  the  body  have  to  have  a  known  capacity.    Perhaps  an  extra  electrode  included  in  the  battery  could 
provide  this  information.    Determination  of  the  remaining  battery  capacity  is  required  for  both 
rechargeable  and  nonre  charge  able  types. 

A.  3.3.6     Pulse  Generators 

Dynamic  testing  of  analog  devices  is  often  done  using  a  flat-topped  pulse,  but  this  pulse  is 
difficult  to  obtain.    The  response  of  a  system  to  an  ideal  flat -topped  pulse  can  be  calculated  if  it  is 
excited  with  an  exponential  shaped  pulse.    This  pulse  is  relatively  easy  to  generate.    Some  applica- 
tions will  still  require  a  good  square  pulse.    To  test  the  vertical  deflection  amplifiers  of  oscillo- 
scopes we  need  a  pulse  with  200  ps  rise  time  and  with  aberrations  on  top  of  the  pulse  of  less  than  1% 
for  10  ns  after  the  comer.    A  repetition  rate  variable  from  50  Hz  to  50  MHz  is  required,  although 
not  necessarily  all  in  one  unit.    Pulse  generators  are  also  used  to  measure  settling  time.    A  cus- 
tomer needs  a  general  purpose  instrument  for  verifying  manufacturers'  specifications  on  devices. 
Ten  nanoseconds  rise  time,  with  30  ppm  aberration  on  the  step  for  10  seconds,  and  a  repetition  rate 
of  0. 1  Hz  to  10  Hz  would  be  sufficient. 

A.  3. 3. 7     Ramp  Generators 

It  was  acknowledged  that  precision  linear  ramp  generators  could  be  used  for  aperture  time 
measurements.    A  ramp  with  100  ppm  linearity  was  considered  adequate.    It  was  mentioned  that 
aperture  time  can  also  be  measured  in  the  frequency  domain  by  special  techniques.    No  one  admitted 
to  any  particular  expertise  in  this  field. 

A.  3 . 3 .  8     Impulse  Te  sting 

Impulse  testing  is  desirable  for  testing  on  line.    Frequency  sweep  methods  (even  using 
ATE's)  require  the  device  to  be  removed  from  service.    Using  impulse  testing,  an  initial  applied 
impulse  can  be  quickly  followed  by  a  cancelling  impulse,  thus  the  device  can  be  tested  without  sig- 
nificantly disturbing  the  system.    Impulse  testing  can  also  be  used  to  calibrate  a  closed  loop  control 
process.    The  problem  is  that  we  are  looking  for  a  small  change  from  standard  behavior  resulting 
from  the  perturbing  impulse.    What  changes  in  the  output  are  evident  from  5%  to  10%  changes  in  the 
parameters  of  the  components  in  the  system?   The  procedure  and  techniques  must  be  investigated 
and  well  documented  to  convince  skeptics  that  impulse  testing  is  reliable.    To  make  full  use  of  this 
technique  the  parameters  of  the  perturbing  pulse  must  be  known  and  described  accurately. 

A.  3.4     Systems 
(F.  Seeley,  Chairman;  R.  Kleimann,  NBS  Recorder;  J.  Morrow,  NBS  Participant) 
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A.  3. 4. 1     Introduction 

The  Systems  Sub-Group  attempted  to  limit  its  consideration  to  those  problems  specific  to  and 
inherent  in  systems  per  se.    It  was  generally  agreed  that  systems  to  be  considered  are  characteris- 
tically assemblies  of  electronic  or  electrical  hardware,  usually  including  some  type  of  software,  and 
designed  for  very  specific  applications.    It  is  common  for  digital  computers  or  other  programming 
equipment  to  be  used  for  rapid  and  reliable  control  of  modern  electrical  test,  measurement,  or  con- 
trol systems.    The  fact  that  systems  are  designed  for  specific  functional  applications  is  as  much  a 
distinguishing  characteristic,  perhaps  more  so,  than  the  fact  that  they  may  contain  some  particular 
set  of  components,  are  expensive,  or  are  large  and  complex. 

It  has  been  a  common  experience  among  systems  designers  and  users  to  find  that,  when  hard- 
ware components  and  software  are  assembled  and  operated  as  a  "system",  problems  exist  which  are 
not  found  when  these  same  components  are  used  individually.    A  new  environment  is  created  within  a 
system  in  which  occur  electrical  interactions  and  interference  among  measurements,  stimuli, 
switches,  controllers,  and  even  commercial  power  sources.    The  problem  facing  the  designer  and 
user  of  systems  is  to  understand  this  new  environment,  and  to  develop  techniques  for  operating  with- 
in or  minimizing  the  deleterious  effects  of  the  systems  environment.    A  related  and  crucial  problem 
concerns  the  ability  to  determine  whether  the  process  or  test  being  conducted  by  a  system  is,  in 
fact,  valid.    The  problems  of  system  environment  and  system  validation  are  two  general  areas  that 
have  been  identified  for  investigation  by  the  Electricity  Division  at  NBS. 

A.  3.4. 2     Performance  Validation:   Static 

The  problem  of  validating  the  steady  state  performance  of  a  system  was  discussed,  and  needs 
established.    Specifically,  transportable  standards  capable  of  providing  steady  state  inputs  and/or 
measurements  to  a  system  at  its  interface  ports  are  needed.    These  standards  will  then  provide  a 
means  of  determining  what  a  system's  steady -state  measurement  capabilities  are  in  its  specific  cli- 
matic, electrical,  and  operational  environment.    If  NBS  were  to  become  involved  in  such  in-system 
verification  through  the  use  of  transportable  electrical  standards,  it  would  be  an  extension  of  the 
Measurement  Assurance  Program  concept  to  automated  systems.    However,  it  was  pointed  out  that  a 
MAP  service  for  ATE  may  be  difficult  to  implement  because  of  the  diversity  of  ATE  systems,  par- 
ticularly in  terms  of  interface  and  software  differences.    The  electrical  quantities  of  principal  inter- 
est for  steady- state  include: 

AC  Voltage:  To  1000  V,  20  Hz  to  .10  MHz, 

uncertainties  1%  to  20  ppm 

DC  Voltage:  To  1000  V,  0. 1%  to  10  ppm 

Resistance:  0. 001  Q  to  100  MO,  0. 1%  to  10  ppm 

Pulse  Characteristics:  Rise,  fall  times  of  10-30  ps; 

exponential  decay,  1%  to  10  ppm  in 
amplitude  uncertainty 

Constant  Current  (DC/AC):  1  nA  to  10  A,  0. 1%  to  50  ppm 

A.  3.4.3     Performance  Validation:   Dynamic 

The  problem  of  chief  concern  is  whether  a  system  will  correctly  perform  its  intended  func- 
tion, e.g.,  reject  bad  parts,  accept  good  ones,  properly  diagnose  faults,  make  measurements  to 
proper  tolerances,  etc.    The  System  Sub-Group  felt  that  inadequate  methods  exist  for  validating 
system  performance  in  the  systems  environment.    Because  of  the  speed  at  which  modern  systems 
operate,  many  time -dependent  and  history -dependent  phenomena  can  no  longer  be  considered 
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negligible  during  system  operation.  The  specific  ways  that  measurement,  stimuli,  switching,  and 
interconnection  components  interact  with  the  outside  world  in  dynamic  tests  is  not  well  understood. 
Needs  include  means  for  proving  that  systems  meet  their  intended  objectives: 

1.  Specifically,  in  order  to  validate  dynamic  performance,  it  was  recommended  that 
"dynamic  physical  and  paper  standards"  be  developed. 

2.  Among  other  things  needed  are  techniques  for  measuring  and  characterizing  the  time 
dependence  of  AC  and  DC  sources  and  references,  as  well  as  components  such  as  resis- 
tors. 

3.  An  additional  need  is  methodology  for  characterizing  source  and  instrument  performance 
in  the  face  of  dynamic  switching  and  dynamic  loading  conditions  found  in  systems. 

4.  Finally,  a  need  was  discussed  for  the  development  of  measurement  techniques  to  be  used 
to  measure  the  pulse  characteristics  found  in  digital  logic  testers.    In  systems  of  this 
type,  many  pulses  are  simultaneously  provided  at  multi-pin  interfaces  and  are  delivered 
at  a  4  MHz  rate.    The  pulse  characteristics  of  interest  include  rise -time,  fall  time, 
amplitude,  overshoot,  skew,  etc. 

A.  3.4. 4     Data  Outliers 

Another  aspect  of  system  performance  validation  was  discussed  as  a  possible  need.     Fre- 
quently, users  of  automatic  systems  will  obtain  data  or  test  results  which  are  unexplainably  bad. 
The  usual  practice  in  this  case  is  to  immediately  repeat  the  test  in  the  hope  that  the  outliers  in  the 
data  will  not  reappear.    If  this  action  succeeds,  many  system  operators  will  completely  disregard 
the  outliers  without  looking  for  any  cause-effect  relationship.    The  Systems  Sub-Group  could  not 
agree  on  any  specific  need,  but  the  frequency  of  such  occurrences  is  such  as  to  make  the  presence  of 
data  outliers  a  disturbing  phenomena.    Perhaps  what  is  needed  is  a  methodology  for  handling  these 
circum  stances . 

A.  3. 4.  5     System  Environment 

There  are  two  strategies  discussed  for  dealing  with  the  problems  of  noise  and  electromag- 
netic interference.  The  first  of  these  strategies  is  to  determine  the  causes,  and  means  of  reducing, 
noise  levels  in  systems.  One  major  area  identified  was  the  need  for  definition  and  characterization 
of  pollution  of  the  power  line,  both  from  external  sources  entering  the  system  environment,  and 
from  effects  of  the  system  itself.  This  definition  and  characterization  could  result  then  in  a  written 
document  of  recommended  specific  practices  and  techniques  such  as  shielding,  guarding,  grounding, 
isolation,  etc.,  which  were  found  to  be  effective. 

A  second  strategy  which  may  be  effective  recognizes  that  noise  can  never  entirely  be  elimi- 
nated and  that  measurements  must  be  made  in  noisy  environments.  Therefore,  it  is  recommended 
that  consideration  be  given  to  a  study  of  system  measurements  and  definition  of  the  limits  or  effects 
of  measurements  made  in  the  presence  of  noise.  One  possible  approach  is  the  generation  of  known 
noise  conditions  to  determine  their  effects  on  system  performance  or  the  effectiveness  of  reduction 
techniques. 

A.  3. 4.  6     Detailed  Session  Notes 

While  results  of  the  Systems  Working  Sub-Group  sessions  have  been  summarized  in  the  pre- 
ceding paragraphs,  the  actual  session  notes  follow,  to  provide  additional  context  and  detail. 

One  system  manufacturer  requires  a  system  test  of  all  possible  error  factors,  to  be  used  in 
both  certifying  system  operation  and  diagnosing  faults.    Another  participant  described  his  areas  of 

38 


responsibility  in  the  crash  testing  of  automobiles.    His  problem,  too,  was  described  as  certifying 
the  operation  of  his  system  before,  during,  and  after  a  test.    The  system'  involves  the  measurement 
of  physical  parameters  during  a  very  short  time,  a  few  seconds  at  most.    The  economic  considera- 
tions are  extreme,  so  the  data  acquired  in  a  test  must  be  usable  and  reliable.    In  addition,  some 
lawsuits  involving  the  real  occurrences  being  simulated  can  run  into  the  millions  of  dollars.    Thus 
the  data  must  be  so  reliable  that  it  can  be  used  in  court.    The  system  described  by  another  manufac- 
turer is  a  redundant  computer,  data  acquisition  system  using  remote  sensors  connected  via  30  baud 
telephone  lines  to  detect  illegal  entries,  etc.    This  system,  too,  requires  some  method  of  certifying 
correct  operation  throughout,  from  the  sensors,  through  A-D  converters,  etc.,  to  the  output.    Also 
needed  is  a  software  algorithm  for  linearizing  the  nonlinear  response  of  the  sensors. 

Another  participant  described  his  needs  in  regard  to  systems  which  monitor  various  parame- 
ters in  power  sub-stations,  calibration  of  customer  meters,  and  instrumentation  for  power  control. 
The  main  problems  are  the  environment  of  a  sub-station  containing  motors,  generators,  turbines, 
high  voltages,  and  currents;  a  need  to  predict  failures  in  various  pieces  of  equipment;  measure- 
ments in  the  UHV  range  (1-2  million  volts);  measurement  of  corona  power  dissipation;  the  need  for 
guidance  in  certifying  software  systems  and  programs,  and  in  verifying  their  correct  operation. 

One  system  manufacturer  stated  that  one  of  his  problems  is  a  lack  of  knowledge  about  what  is 
available  at  NBS.    He  suggested  that  NBS  should  supply  standards  for  use  in  ATE's.    He  elaborated 
by  saying  that  the  standards  now  supplied  cannot  be  used  in  ATE's  at  a  typical  ATE  speed  because 
only  a  long-term  value  of  the  standard  is  supplied.    What  is  needed  is  a  dynamic  standard  (possibly 
programmable)  to  be  interfaced  with  the  ATE,  and  checked  as  a  reference  during  a  normal  run  of  the 
system.    It  developed  later  that  the  economic  motivation  for  this  type  of  standard  is  that  after  spend- 
ing hundreds  of  thousands  of  dollars  for  an  ATE,  it  is  very  expensive  to  have  to  wait  for  the  stan- 
dards to  settle  during  calibration  or  to  have  to  tear  down  the  ATE  and  send  parts  to  a  standards 
laboratory  for  calibration.    It  is  necessary  to  have  standards,  usable  in  the  environment  of  the  sys- 
tem, with  known  values  and  uncertainties  as  a  function  of  time.     For  example,  the  value  and  uncer- 
tainty of  a  standard  resistor,  20  ms  after  it  is  switched  into  the  measuring  system,  must  be  known 
in  order  for  it  to  be  useful  as  a  reference  in  a  system  where  perhaps  100  measurements  are  taken 
per  second.    Another  important  need  for  such  a  dynamic  standard  is  for  AC  references  with  settling 
times  in  the  range  of  100  /is  to  1  ms,  with  accuracies  of  about  0. 1%. 

Further  general  discussion  showed  that  the  need  to  know  what  the  system  is  doing  is  very 
great.    That  is,  a  way  of  verifying  system  operation  must  be  found  in  order  to  establish  credibility 
in  the  system.    It  was  generally  agreed  that  the  standard  should  be  brought  to  the  system,  rather 
than  vice  versa.    The  problem  of  what  to  do  about  systems  which  do  not  measure  but  rather  acquire 
and  process  data  (such  as  security  systems,  scanning  microscopes,  etc. )  was  discussed.    One  pos- 
sibility mentioned  was  that  NBS  might  develop  internal  references  to  be  used  within  a  system  of  this 
sort. 

It  was  suggested  that  NBS  determine  the  effect  on  specifications  of  instruments,  components, 
and  standards  when  placed  in  a  typical  use-type  environment  rather  than  a  standards  laboratory- 
type  environment  where  the  specifications  are  normally  measured.     For  example,  specifications 
may  require  +  1  C,  but  the  equipment  may  be  used  where  there  is  no  means  of  controlling  the  tern  - 
perature.    The  role  of  NBS  in  this  effort  would  probably  be  to  develop  techniques  for  testing  instru- 
ments in  a  use-type  environment  rather  than  doing  the  actual  testing.    One  way  NBS  could  do  this 
would  be  to  provide  sufficient  additional  data  with  reports  of  calibration  to  permit  the  user  to  predict 
the  changes  in  performance  to  be  expected  from  the  changes  in  environmental  conditions. 

The  standardization  of  software  systems  was  discussed.     For  the  most  part,  economic  con- 
siderations can  sometimes  force  standardization  of  software.    The  NBS  Request  for  Proposal  (RFP) 
for  198  minicomputer  systems  may  have  already  caused  some  interested  manufacturers  to  modify 
their  systems  to  comply  with  the  specifications  of  the  RFP,  according  to  an  NBS  participant.    It  was 
generally  felt  that  some  way  of  verifying  software  systems  was  needed.    It  was  suggested  that  NBS 
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develop  a  software  package  of  calibration  techniques,  guidelines,  and  a  list  of  pitfalls  in  the  use  of 
ATE  measurements. 

Additional  discussion  concerned  the  problems  of  validating  system  operation.    The  problem 
of  interaction  effects  was  introduced.    These  effects  are  brought  about  when  one  part  of  a  system 
causes  an  error  in  another  part.    Some  examples  given  were:    Stimulus/measurement  interaction; 
history-dependent  phenomena,  such  as  stored  energy  in  a  switch;  and  time -dependent  phenomena.    It 
was  felt  that  NBS  could  help  with  these  problems  by  developing  transportable  standards  usable  on  a 
system  in  a  system  environment.    A  list  of  various  standards  was  made  by  obtaining  the  preferences 
of  the  members  of  the  group.     For  AC  voltages,  standards  up  to  10  MHz,  up  to  1000  V,  with  sine, 
square,  ramp  and  sawtooth  waveforms,  rise  and  delay  times  of  10  to  30  ps  and  accuracies  ranging 
from  1%  to  10  ppm  were  mentioned  as  desirable.    For  DC  voltages,  up  to  1000  V,  0. 1%  to  10  ppm. 
For  resistance,  1  mQ  to  100  MO,  0. 1%  to  10  ppm.     For  power,  60  Hz,  0. 1  W  to  100  W,  1%.    Also 
listed  were  constant  current  sources,  DC  and  AC,  ranging  from  lnAto  10  A  with  accuracies  of  0. 1% 
to  0.  005%  and  a  pulse  standard  with  a  rise  time  of  10  to  30  ps  and  an  exponential  decay  thereafter. 

The  problem  of  needing  standards  and  references  operating  at  ATE  speeds  was  brought  up 
once  more.    Again  it  was  felt  that  dynamic  standards,  wherein  the  values  and  uncertainties  as  a 
function  of  time  are  known,  are  needed.    It  was  felt  that  NBS  could  develop  the  techniques  for  mea- 
suring time  dependence  of  AC  or  DC  sources  and  references.    It  was  also  felt  that  NBS  could  charac- 
terize and  measure  the  effects  of  dynamic  load  conditions  and  switching  on  sources  and  measuring 
devices. 

In  reference  to  interaction  effects,  one  participant  described  two  of  his  problems  regarding 
the  automatic  testing  of  digital  components.    The  first  problem  was  in  detecting  faulty  items.    Some 
items  were  called  faulty  by  the  system  when,  in  fact,  they  were  not,  because  of  interactions  of  one 
measurement  on  the  following  measurement,  a  history -dependent  interaction.    Some  other  items 
were  stressed  so  much  by  the  test  that  they  were  more  likely  to  fail  when  put  into  use,  while  for 
certain  other  items  the  tests  actually  caused  failures,  again  due  to  succeeding  test  interactions. 

Another  problem  concerned  digital  logic  testers  in  which  a  large  number  (300)  of  high  rep 
rate  (4  MHz)  pulses  are  applied  to  an  item,  and  the  characteristics  (rise  and  fall  time,  amplitude, 
overshoot,  skewness)  of  a  large  number  of  output  pulses  must  be  measured.    It  was  felt  that  NBS 
might  be  able  to  develop  techniques  for  these  types  of  measurements,  taking  into  account  the  inter- 
action effects  of  the  pulses. 

The  other  major  system  problem  discussed  at  this  session  was  electromagnetic  interference 
(EMI)  from  various  sources:   Power  transformers,  power  lines,  etc.    Most  interference  problems 
stem  from  what  was  described  as  "power  line  pollution",  i.e. ,  transients,  spikes,  etc. ,  which 
cause  the  AC  power  line  to  deviate  from  a  60  Hz,  117  V  sine  wave.    It  was  felt  that,  in  regard  to 
EMI,  NBS  should  perhaps  try  to  characterize  and  define  "power  line  pollution",  develop  methods  and 
techniques  of  minimizing  the  generation  of  this  "pollution"  by  isolating  "polluters"  from  the  power 
line,  and  recommend  practices  for  shielding,  guarding,  grounding  and  isolating  equipment  which 
would  be  sensitive  to  the  "pollution". 

It  was  also  felt  that  NBS  could  define  the  limits  of  or  effects  on  measurements  made  in  a 
"noisy"  environment.    It  also  may. be  possible  to  define  or  generate  a  standard  "noisy"  environment 
for  testing  purposes. 

The  rationale  behind  the  desirability  of  NBS  setting  up  definitions,  measurement  techniques, 
etc. ,  is  that  it  will  provide  marketplace  equity  and  better  define  contractual  obligations  between  sys- 
tem manufacturers  and  their  customers.    For  example,  the  manufacturer  or  customer  may  not  be 
aware  of  or  cannot  be  aware  of  the  present  and/or  future  environment  in  which  the  system  will  be 
operated.    This  type  of  situation  caused  a  $320,  000  lawsuit  in  one  case.    If  there  had  been  a  defined 
"noisy  environment"  the  suit  may  have  been  avoided. 
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The  problem  of  data  outliers  or  "glitches"  was  discussed.    What  do  you  do  about  "glitches"? 
How  do  you  identify  "glitches"  as  opposed  to  true  values,  etc.  ?   It  was  believed  that  NBS  might  be 
able  to  devise  a  methodology  for  identifying  and  handling  these  data  outliers. 

The  philosophy  of  using  idealized  stimuli  in  tests  was  discussed.    It  was  felt  that  the  use  of 
idealized  or  nearly  perfect  square  waves,  for  example,  to  test  a  component  which  will  be  used  in 
equipment  in  which  the  square  waves  are  not  anywhere  near  perfectly  flat,  seems  to  be  fallacious. 

Other  topics  briefly  mentioned  were:   The  problem  of  dielectric  absorption,  and  the  calibra- 
tion of  capacitors  at  frequencies  higher  than  now  supplied.    Also,  many  members  of  the  Workshop 
seemed  to  be  unaware  and,  indeed,  find  it  hard  to  believe  that  NBS  is  not  a  regulatory  agency  of  the 
Federal  government.    In  this  same  vein,  it  was  also  suggested  that  NBS  could  perform  a  useful  serv- 
ice in  defining  what  constitutes  traceability. 

A.  4     Notes  on  Sub-Group  Chairmen  Oral  Presentation  at 
Closing  Plenary  Session,  with  Associated  Discussion 

The  next  four  sections  are  verbatim  transcriptions,  edited  by  the  moderator  only  for  style, 
of  the  oral  presentations  of  the  four  working  sub-group  chairmen  at  the  closing  plenary  session. 
The  questions  and  answers  which  followed  each  of  the  presentations  are  also  included. 

A. 4.1     Components  and  Parameter  Measurements 
(Presented  by  E.  Boeckmann) 

There  were  discussions  on  high-speed  measurements  on  capacitors,  including  the  problem  of 
dielectric  absorption.    People  would  like  to  see  effects  less  than  1000  ppm,  and  possibly  as  low  as 
10  ppm  over  a  range  of  capacitance  of  10  pF  to  1  juF. 

Also,  there  was  a  feeling  that  the  older  methods  of  characterizing  capacitors  in  terms  of 
dielectric  constant  and  dissipation  factor,  and  so  on,  are  now  somewhat  inadequate.    We  are  more 
interested  in  the  response  to  pulses  of  voltage,  usually  in  the  1  to  10  volt  area,  to  10  MHz  with  a 
50  percent  duty  cycle,  as  a  typical  set  of  criteria. 

Phase  angle  measurements  may  be  of  some  interest  in  this  area,  and  there  may  be  a  need  for 
material  studies  in  terms  of  what  types  of  dielectric  materials  could  contribute  an  improvement,  or 
what  types  of  new  materials  might  be  developed.    There  was  a  discussion  of  the  type  of  standard 
capacitor,  air  dielectric,  or  what  have  you,  that  could  be  used  as  a  standard. 

We  divided  resistors  into  three  different  categories,  100  Cl  to  1  Mf2,  1  Mf2  to  1  G€l,  and  1 
GO,  to  10      Q  or  10  teraohms  (TO),  if  you  want  to  use  that  terminology.    In  the  100  Q  to  MD  range, 
there  is  more  interest  in  the  pulse  response  characteristics,  more  so  than,  say,  specification  of  DC 
parameters  or  AC  characteristics.    For  example,  what  would  be  the  effect  of  10  MHz  at  50%  duty 
cycle?   In  that  case,  people  are  generally  looking  for  0. 1%  accuracy  as  opposed  to  1%,  or  at  100 
MHz  we  are  looking  for  possibly  1%. 

The  A  to  D  conversion  people  are  interested  in  very  high  stability  resistors,  preferably  film 
types  and  there  apparently  is  some  feeling  that  film  resistors  sometimes  leave  something  to  be 
desired  in  terms  of  stability  for  very  high  precision  ladders. 

Also,  there  is  a  feeling  that  there  is  a  need  for  calibration  services  on  binary  values.    Also, 
possibly  a  need  for  the  calibration  of  network  ratios  for  voltage  dividers  to  accuracies  of  at  least  1 
ppm.    Some  of  the  NBS  work  we  saw  yesterday  in  the  cryogenic  area  would  indicate  a  capability  for 
very  precise  calibration  of  voltage  dividers. 
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In  the  1  MO  to  1  GQ  area,  there  was  a  requirement  for  500  ppm  accuracy  and  at  least  one 
user  was  looking  for  10    Q,  or  10      Q  at  0. 35%  accuracy.    Apparently,  this  type  of  calibration  can 
be  obtained  elsewhere  at  the  present  time. 

We  discussed  noise  in  some  detail,  but  more  from  the  point  of  view  of  the  availability  of  a 
precision  noise  standard,  or  noise  generator  versus  different  ranges  of  frequency  from  10  Hz 
through  the  audio  spectrum  on  up  to  500  MHz. 

We  discussed  (a)    a  paper  standard  and  (b)   physical  standards,  possibly  the  construction  of  a 
precision  white  noise  or  gaussian  noise  generator,  perhaps  using  a  computer  to  generate  truly  ran- 
dom impulses  for  the  computer  measurement  of  such  a  source  with  calculations  against  a  gaussian 
formula.    Also  discussed  were  methods  of  measurement  of  noise  standards  for  purposes  of  calibra- 
tion.   Applications  included  hearing  aids,  plus  calibration  of  components  such  as  resistors  and 
diodes,  where  we  would  like  to  have  a  standard  noise  source  for  very  low-level  noise. 

Also  discussed  in  somewhat  more  general  terms  was  the  medical  implant  area  in  terms  of 
the  reliability  testing  of  these  devices  against  a  5-year  life  requirement.    In  other  words,  how  good 
is  a  particular  lot  of  devices,  should  they  be  sampled,  put  on  an  accelerated  test,  or  should  we 
depend  on  a  manufacturer's  claims?    For  example,  in  the  pacemaker  area,  there  is  a  question  con- 
cerning the  amount  of  current  that  is  required  to  consider  the  device  in  an  operational  mode.     Fig- 
ures ranged  from  1  mA  down  to  25  /iA. 

There  is  a  class  of  devices  which  we  could  call  stimulators,  including  pacemakers,  dorsal 
column  stimulators,  phrenic  nerve  stimulators,  and  so  on,  which  should  perhaps  fall  under  an  elec- 
trical or  system  characterization  standard  or  specification.    In  the  area  of  hospital  equipment, 
there  may  be  a  need  for  calibration  of  such  devices  as  clinical  analyzers,  or  other  monitoring 
devices.    There  was  more  discussion  on  hearing  aids,  with  possible  interest  in  characterization  in 
terms  of  frequency  response  and  distortion  levels,  at  a  noise  figure  of  5%  to  10%. 

Miscellaneous  topics  include  possible  standards  for  temperature  coefficient  calibration  for 
resistors  or  capacitors.    There  is  also  apparently  a  requirement  for  inductive  measurements  of  10 
to  50  ppm  accuracy  at  200  Hz  to  2  kHz  in  the  100  /iH  to  10    H  range.    There  was  a  discussion  of  the 
need  for  a  general  study  by  NBS  of  pulse  techniques,  pulse  transmission,  and  response  of  compo- 
nents to  pulses.    Measurements  of  low  resistance  in  the  10      Q  area  seems  to  be  of  some  impor- 
tance, but  at  low  accuracies. 

A  brief  discussion  was  held  on  the  problems  of  the  interfacing  of  transducers  to  systems,  in 
terms  of  the  interaction  between  the  Electricity  Division  and  the  other  transducer  calibration  facili- 
ties at  NBS,  but  nothing  specific  in  the  way  of  suggestions. 

There  was  some  discussion  on  the  need  for  high-speed  resistor  calibration,  as  applied  to 
laser  trimming.    We  may  trim  several  resistors  a  second  and  therefore  we  need  to  be  able  to  have  a 
system  which  will  measure  the  resistor  accurately  under  these  high-speed  conditions. 

Q:    Was  there  any  discussion  of  not  just  the  use  of  white  noise,  but  the  characterization  of 
noise  from  the  devices,  the  characterization  of  its  distribution  if  it  is  non-gaussian? 

A:    Following  the  meeting  there  was  discussion  concerning  the  possibility  of  some  device 
noise  having  a  non-gaussian  distribution,  in  which  case  erroneous  calibrations  could  be  made,  or 
other  types  of  problems  could  arise.    Actually,  I  believe  noise  characterization  could  be  a  fruitful 
area  of  investigation  in  terms  of  the  development  of  precision  ways  of  determining  whether  or  not 
the  distribution  is  really  gaussian,  and  what  is  the  amplitude  versus  frequency,  and  distribution. 
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Q:   Was  there  any  discussion  on  the  need  for  improved  accuracy  in  power  standards?   We 
have  seen  the  digital  wattmeter,  for  example,  and  other  techniques,  generally  in  the  region  of  0.01%, 
or  thereabouts. 

A:   There  was  a  discussion  in  the  area  of  measurement  of  power  factor.    I  think,  at  least, 
one  gentleman  was  talking  about  0. 01%  power  factor  measurements  and  he  was  upset  at  not  being 
able  to  get  a  good  phase  angle  meter.    He  felt  that  he  would  not  have  any  confidence  in  its  calibra- 
tion.   So,  this  may  be  an  area  of  interest. 

Q:    I  would  suggest  that  the  Bureau  consider  a  factor  of  10  improvement  in  what  exists  today 
for  power  meter  calibrating  purposes  to  20  ppm,  say. 

Presently,  there  are  transducers  and  power  meters  on  the  market  that  have  specs  in  the 
order  of  0.01%.    In  order  to  ascertain  whether  or  not  these  are  real,  we  would  like  the  Bureau  to 
say,  "Yes,  that  is  a  valid  number  and  we  know  because  we  are  10  times  better  than  that.  "  As  of 
now,  you  can  basically  say,  "We  think  it  is  because  we  are  about  as  good.  "    It  would  seem  that  would 
be  an  area  where  some  additional  development  might  take  place.    On  the  power  factor  and  phase 
measurements,  I  would  be  particularly  concerned  with  the  increasing  effect  of  harmonic  distortion 
and  how  that  might  be  dealt  with. 

Q:   (NBS  Participant):     I  would  like  to  make  some  comments  about  some  of  the  calibrations 
that  you  mentioned.    We  do  have  the  capability  for  somewhat  improved  accuracy  over  those  figures 
that  were  mentioned  in  the  high  resistance  area.    In  particular,  we  can  do  up  to  about  10      Q  with 
an  uncertainty  no  greater  than  35  parts  per  million.    We  do  not  do  quite  that  well  for  10     ,10     , 
10     ,  and  10      Q,  but  I  think  that  we  could  meet  that  0.35%  uncertainty  figure  that  was  quoted  as  a 
need.    The  reason  that  we  do  not  normally  give  these  high  accuracies  on  our  regular  calibrations  is 
that  the  resistors  that  we  receive  usually  do  not  warrant  it.    They  have  such  high  voltage,  tempera- 
ture, and  humidity  coefficients  that  it  is  pointless  to  put  that  kind  of  accuracy  on  the  calibration. 

One  other  point.    We  do,  and  traditionally  have,  a  temperature  coefficient  measurement,  at 
least,    on  standard  resistors  and  we  do  calibrate  resistance  standards  at  the  milliohm  and  sub- 
milliohm  levels. 

Q:    I  should  like  to  comment  that  a  very  logical  extension  of  the  Division's  sampling  watt- 
meter program  that  would  make  a  lot  of  sense,  might  be  going  to  much  higher  frequencies,  taking 
the  samples  not  all  in  the  same  time  frames,  but  using  sampling  technology,  perhaps  only  having  to 
improve  the  sample -and-hold  capability.    That  looks  like  a  very  logical  program  to  extend.    There 
are  meters  available  in  just  about  every  frequency  range  you  can  imagine,  for  which  one  might  like 
to  have  better  capability  for  precision  calibration. 

A.  4. 2     Signal  Conditioning  and  Data  Conversion 
(Presented  by  D.  Ludwig) 

Most  participants,  at  one  time  or  another,  made  it  into  Session  3  where  we  talked  about  a 
long  list  of  possible  things  where  NBS  might  be  useful  to  industry. 

At  the  beginning  we  discussed  the  general  question  of  characterization  and  definition,  versus 
measurement.    In  the  measurement  discussions,  we  tried  to  home  in  on  the  measurement  problems 
and  the  places  where  equipment  could  be  really  improved.    But  there  was  agreement  that  it  would  be 
impossible  to  measure  parameters  that  have  not  been  defined.    If  these  definitions  can  be  achieved, 
then,  either  on  a  de  facto  basis,  or  otherwise,  they  could  become  considered  as  standard,  and  at 
least  we  would  then  have  a  common  language  for  performance  characteristics. 

We  spent  a  fair  amount  of  time  talking  about  settling  time,  probably  because  it  seems  to  be 
the  one  that  there  is  the  least  industrial  capability  to  measure  of  all  of  the  parameters  discussed. 
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There  is  quite  a  disparity  between  the  capability  in  industry  to  make  this  measurement,  be- 
tween manufacturers  and  users.    It  was  pointed  out  that  the  users  are  pretty  much  at  the  mercy  of 
the  manufacturers  with  respect  to  being  able  to  make  this  kind  of  a  measurement,  or  to  certify  this 
kind  of  performance.     Even  the  manufacturers  themselves  could  use  greater  capability  in  terms  of 
making  really  precision  settling  time  measurements. 

Settling  time,  of  course,  is  a  very  complex  characteristic,  requiring  both  a  voltage  and  a 
time  measurement  to  be  made.    Having  these  two  degrees  of  freedom,  makes  it  more  difficult  to  talk 
about. 

There  was,  however,  very  definitely,  a  general  consensus  that  all  of  us  would  appreciate  the 
efforts  that  the  Bureau  might  put  into  learning  about  helping  us  to  define  settling  time,  and  learning 
how  to  measure  it  to  higher  and  higher  levels  of  precision. 

There  are  two  parts  to  the  problem  of  measuring  settling  time.     First  of  all,  in  the  general 
case,  in  order  to  measure  the  result  of  a  step  input  to  a  system  one  must  have  a  sufficiently  clean 
forcing  function.    So,  the  matter  of  deriving  the  forcing  function  that  is  suitably  clean  to  make  the 
measurements  is  the  first  part  of  the  problem.    The  characteristics  of  this  source  would  be  a  flat 
top,  which  would  have  precision  levels,  probably  on  a  DC-coupled  basis,  and  would  typically  need  to 
be  able  to  achieve  settling  time  itself  to  0.  01%  with  20  ns  a  good  target  to  shoot  for.    I  think  that  it 
was  generally  felt  that  if  20  ns  could  not  be  met,  that  there  would  still  be  a  lot  of  use  for  50  ns,  or 
even  100  ns. 

Precision  at  the  top  of  the  pulse  would  need  to  be  0.  01%  as  a  minimum. 

The  rise  time  of  the  pulses  would  have  to  be  well  behaved  and  well  controlled,  and  perhaps 
adjustable.    The  step  waveform  could  not  have  any  overshoot,  or  if  it  had  any  it  would  have  to  be 
fairly  minor  and  well  understood,  or  else  the  resulting  measurement  would  begin  to  look  more  like 
the  impulse  response  than  the  step  response  of  the  product  being  looked  at. 

The  precision  levels  would  have  to  be  adjustable  because  there  is  a  wide  range  of  needs  in 
terms  of  input  signal  steps  that  we  would  be  interested  in  having  these  measurements  made  for,  and 
both  polarities  of  step  would  have  to  be  available.    So,  effectively,  needed  is  a  step  where  both  the 
beginning  and  ending  levels  are  more  or  less  infinitely  adjustable  with  very  good  precision. 

The  measurement  problem  involves,  once  having  forced  this  product  under  test  with  a  step, 
measuring  its  output  behavior.    In  particular,  we  are  looking  for  what  we  loosely  call  settling  time 
in  terms  of  how  long  does  it  take  the  output  to  achieve  an  error  from  final  value  that  is  within  some 
defined  percentage.    0.  01%  and  20  ns  are  good  goals,  but  there  is  a  wide  need  for  0. 1%  measure- 
ments; and  even  0.  005%  measurements  are  certainly  a  desirable  type  number. 

The  measuring  capability  would  have  to  have  the  ability  to  do  this  test  without  the  crutch  of 
the  perfect  input  pulse  for  subtraction  purposes,  for  cases  in  which  the  input  is  merely  a  digital 
word  change,  and  the  output  is  an  analog  step  which  has  to  be  measured  independently  without  the  use 
of  the  perfect  analog  step. 

The  application  for  this  kind  of  measurement,  of  course,  is  to  just  about  everything  you  can 
think  of.  To  list  a  few,  certainly,  amplifiers  of  all  sizes  and  shapes  to  DAC's  to  ADC's  to  sample- 
and-holds,  multiplexers,  resistors,  basically  to  the  whole  wide  range  of  analog  processing  devices 
and  interface  units. 

The  concept  of  measurement  to  within  a  given  percent  of  final  value  deserves  comment,  be- 
cause there  is  a  serious  question  as  to  what  constitutes  final  value,  or  when  do  you  measure  final 
value?    In  general,  a  final  value  at  the  end  of  1  second  might  be  different  from  what  was  measured 
after  only  a  millisecond.    There  have  to  be  some  definitions,  and  judgments  regarding  methodology. 
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It  was  felt  desirable,  if  possible,  to  consider  a  transportable  standard  for  settling  time. 

The  second  topic,  leaving  settling  time,  would  be  the  question  of  characterization  of  data  con- 
verters.   There  was  some  divergence  of  opinion  regarding  whether  this  was  a  useful  activity  for 
NBS,  and  whether  or  not  it  would  be  useful  for  NBS  to  set  up  testing  definitions  that  might  be  con- 
strued as  being  standards,  whether  they  be  formal  or  de  facto  standards.    There  is  some  concern 
that  the  existence  of  these  standards  would  result  in  holding  back  progress,  perhaps,  at  a  point  in 
time  when  we  are  still  in  an  early  stage  of  A  to  D  and  D  to  A  converters,  with  the  possibility  that  it 
might  become  difficult  to  change  these  standards,  and  we  might  end  up  stuck  with  them.    Better 
approaches  might  come  along,  and  there  might  be  resistance  to  change. 

Another  consideration  was  the  reluctance  of  manufacturers  to  be  hamstrung  by  procedures 
that,  even  though  making  sense  at  NBS's  laboratory,  might  well  be  the  wrong  thing  to  do  on  a  produc- 
tion line.    If  the  customers  got  used  to  the  idea  of  such  a  standard  and  began  to  use  it  in  requiring 
production  tests  of  this  type,  this  might  not  be  economically  justifiable. 

The  ultimate  consensus  was  that  this  was  an  idea  where  we  ought  to  proceed  with  care,  that 
there  certainly  is  so  much  confusion  in  the  specifications  that  there  is  a  lot  of  work  needed,  that  NBS, 
in  order  to  make  measurements  in  the  area,  will  have  to  make  some  determinations  of  what  these 
specs  mean  and  that  is  fine  as  long  as  we  can  find  some  way  to  have  everybody  understand  it. 
Results  may  not  be  sensible  tests  for  production  environments,  but  a  test  on  an  A  to  D  converter  that 
is  correct  and  sensible  for  a  laboratory  evaluation  of,  let  us  say,  a  prototype  or  a  characterization 
of  a  product,  is  not  necessary  and  not  even  a  sensible  thing  to  do  on  a  production  basis.    On  a  pro- 
duction line  there  is  a  lot  of  prior  knowledge  about  how  the  converter  has  been  manufactured,  which 
guarantees  certain  kinds  of  performance  will  exist  assuming  other  kinds  of  parameters  have  been 
measured.    This  kind  of  test  program,  or  this  kind  of  definition  of  parameters  and  measurements 
that  could  be  defined  here  would  be  very  sensible  and  useful,  but  ought  not  to  be  tied  to  the  produc- 
tion basis. 

One  of  the  questions  that  arose  was,  if  you  are  going  to  measure  A  to  D  converters,  how 
many  transitions  should  you  measure?   In  a  10-bit  converter  there  are  1024  transitions  by  one  man's 
definition,  but  this  comprises  1024  steps,  and  you  can  change  from  any  of  these  to  any  other,  that  is 
a  1024  factorial  thing  and  obviously  some  very  large  amounts  of  data  could  be  collected.    That  is  only 
on  a  10-bit  converter.    It  gets  staggering  to  think  about  it  at  16  bits. 

There  was  some  discussion  about  using  a  dither  technique,  varying  the  input  to  an  A  to  D4  con- 
verter on  an  AC  basis,  to  make  precision  differential  linearity  measurements. 

The  important  consideration  for  both  DAC's  and  ADC's  is  relative  accuracy,  as  compared  to 
absolute  accuracy.    There  are  a  large  number  of  situations  and  applications  for  converters  in  which 
the  relative  accuracy,  or  the  linearity,  of  the  converter  is  important,  and  the  absolute  numbers  are 
much  less  meaningful.    So,  there  needs  to  be  some  reasonable  basis  for  separating  the  absolute 
from  relative  performance. 

It  was  also  agreed  that,  in  the  area  of  data  converters,  there  are  a  great  number  of  charac- 
teristics that  are  important  that  are  not  now  specified  at  all,  and  we  discussed  some  of  these.    Such 
things  as  the  dynamic  characteristics  of  the  input  impedance  on  an  ADC  or  a  multiplying  DAC,  where 
the  specs  say  5  kfi,  but  that  5  kfi  is  doing  some  very  strange  things  versus  time.    As  another  exam- 
ple, glitches  are  a  characteristic  of  D  to  A  converters  that  is  only  beginning  to  really  surface. 
Their  measurement  is  still  not  well  established  or  well  defined. 

Noise  or  uncertainty  in  the  transition  points  on  an  analog-to-digital  converter  is  another. 
Some  analog-to -digital  converters  have  very  clean  transitions,  and  others  have  very  noisy  ones. 
You  can't  tell  that  by  reading  the  specs. 
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These  are  examples  of  a  few  out  of  many  areas  that  could  well  be  surfaced  and  talked  about 
and  looked  at  and  measured. 

There  may  be  an  interesting  parallel  tq  a  similar  question  that  arose  in  1954,  in  connection 
with  calibration  and  characterization  of  ratio  transformers.    The  manufacturers  had  greater  capa- 
bility than  the  Bureau  to  make  the  measurements,  and  there  were  great  debates  and  discussions  be- 
tween buyers  and  sellers  as  to  whether  the  units  really  met  their  specs.    It  was  a  very  similar  situa- 
tion, and  we  all  know  that  the  Bureau  took  hold,  learned  how  to  do  these  things,  finally  moving  well 
ahead  of  industry  with  their  capability  as  time  went  along.    This  was  ultimately  to  everyone's  bene- 
fit, resulting  in  a  much  sounder  business  base  for  the  people  building  a  product  and  the  people  using 
it. 

A  similar  consensus  exists  that  the  sample -and -hold  is  another  device  that  needs  to  be 
defined,  and  for  which  measurements  need  to  be  improved.    Acquisition  time,  which  is  similar  to  a 
settling  time  measurement,  needs  to  be  worked  on,  with  requirements  on  the  order  of  0.  01%  and  20 
ns.    That  would  give  us  some  margin  today.     For  aperture  uncertainty,  we  are  talking  about  100  ps 
or  better,  and  that  is  quite  an  involved,  difficult  measurement.    Today,  it  is  largely  calculated  in 
industry  rather  than  measured.    We  might  get  some  big  surprises  if  we  could  make  the  measure- 
ments correctly. 

Special  generators  to  make  measurements  on  sample -and -holds  were  discussed,  like  preci- 
sion ramp  generators,  including  ramp  generators  that  make  precision  corners.    It  could  be  a  very 
useful  device  if  you  have  a  suitably  precise  one,  for  learning  what  sample -and -holds  do  when  trying 
to  sample  the  corner. 

Also  of  importance  is  the  memory  effect  of  the  sample -and -hold  device.    This  gets  back  to 
dielectric  absorption,  but  the  measuring  technology  needs  to  be  applied  also  at  the  subsystem  modu- 
lar level  as  well  as  at  the  component  level,  so  that  we  can  see  to  what  extent  the  answer  given  by  the 
sample -and -hold  is  dependent  upon  its  previous  history.    It  is  in  the  tens  of  nanoseconds  and  hun- 
dreds of  nanoseconds  region  that  interest  exists. 

For  multiplexers,  too,  more  definitions  are  needed.    Certain  characteristics  are  not  well 
defined  or  well  measured.    Included  is  dynamic  input  impedance,  particularly  for  make-before- 
break  multiplexers,  which  have  a  terrible  input  impedance  characteristic  inasmuch  as  the  sources 
get  shorted  together  during  transitions.    This  is  largely  an  unknown,  unspecified,  unmeasured  phe- 
nomenon, which  usually  comes  as  a  large  surprise. 

Additional  areas  include  settling  time  for  multiplexers,  the  whole  gamut  of  analog  dynamic 
measurements,  and  cross  coupling  between  off  and  on  channels.    There  is  a  lot  of  work  that  could  be 
done  to  do  a  better  job  of  characterizing  multiplexers.    Multiplexers  are,  erroneously,  often  treated 
like  perfect  switches. 

Data  acquisition  systems  could  be  treated  as  a  whole.    Some  very  interesting  things  could  be 
done  using  frequency  domain  analysis. 

A  consensus  also  exists  that  the  area  of  non-linear  functions  is  one  where  the  definitions  of 
measurements  and  definitions  of  parameters  are  pretty  diverse  from  manufacturer  to  manufacturer. 
It  is  not  clear  when  the  spec  says  0. 1%,  or  1%,  over  what  range  of  inputs  that  applies.    This  is  an 
area  where  some  good,  sound  organization  would  help  to  try  to  get  all  these  definitions  straightened 
out. 

In  the  area  of  RMS,  there  also  is  a  need  for  improved  measurements  for  being  able  to  do  the 
experimental  characterization,  particularly  in  the  presence  of  complex  waveforms  with  significant 
peak  factors. 
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There  is  a  significant  need  to  study,  measure,  and  characterize  some  of  the  non-gaussian 
type  noise  generated  by  the  circuits,  things  like  popcorn  noise,  transition  noise  in  analog-to -digital 
converters  and  chopper  noise  in  chopper  stabilized  amplifiers. 

On  the  subject  of  long-term  stability,  we  talked  at  some  length  about  the  very  great  desire  on 
all  of  our  parts  for  NBS  to  help  in  an  area  that  we  are  all  concerned  about.    This  is  the  question 
about  the  A  to  D  converter  that  is  14  bits  today.    Will  it  be  14  bits  next  year?    First,  of  course, 
come  the  components.    If  we  knew  or  could  predict  more  about  what  they  will  do  in  the  next  year  or 
two,  or  could  extrapolate  from  short-term  measurements,  we  could  determine  what  we  might  expect 
for  performance  of  the  functional  circuit. 

So,  the  suggestion  is  that  it  would  be  very  useful  to  have  some  very  carefully  taken  data  on 
long-term  stability  of  resistors,  inductors,  capacitors,  V     ,  h      ,  resistance  ratios,  V_Q  for 
FET's,  temperature  coefficients  and  so  forth  versus  time,  and  correlating  results  against  predic- 
tions based  on  burn-in  temperature  cycling. 

At  this  point  in  time,  manufacturers  try  to  take  account  of  stability  with  some  conservatism 
in  design,  plus  careful  selection  of  processes  and  components;  but  nobody  has  much  real  assurance 
about  what  is  going  to  happen  next  year. 

There  is  a  definite  need  for  laboratory  level  D  to  A  converters  that  could  be  useful  for  both 
testing  DAC's  by  comparison,  as  well  as  ADC's  with  10  ppm  type  accuracy.    Such  "standard"  DAC's 
ought  to  have  a  floating  output,  to  at  least  +  1  volt,  in  order  to  avoid  grounding  problems. 

There  was  a  brief  discussion  about  the  mechanisms  of  communication  between  NBS  and  indus- 
try.   This  came  up  with  respect  to  a  particular  program  that  had  been  conducted  on  bonding  and  the 
reliability  of  bonds.    There  was  some  discussion  as  to  how  the  industry  at  large  would  get  to  know 
about  a  program  of  this  type  that  was  carried  on  at  NBS.    The  work  on  bonding  might  have  saved  a 
lot  of  us  some  money  had  we  known  about  it. 

Voltage-to-frequency  converters  have  gone  from  an  instrument -type  of  product  to  more  of  a 
functional  circuit.    Not  really  much  has  been  done  on  characterizing  linearities,  dynamic  range, 
etc. 

Ideally,  it  would  be  desirable  to  be  able  to  temperature  test  all  of  the  parameters  discussed. 
It  may  be  that  in  any  new  instrumentation  pursued,  NBS  should  include  the  ability  to  control  the  envi- 
ronment, as  part  of  the  initial  design.     If  you  build  a  settling  time  tester,  it  is  too  late  to  wait  until 
after  you  are  done  to  decide  that  you  want  to  vary  the  environment  on  the  unit  under  test.    Indeed, 
definitions  for  temperature  coefficient  itself  could  definitely  stand  some  standardization. 

Also  discussed  was  wafer  probe  testing,  and  the  desire  to  be  able  to  do  all  these  kinds  of 
tests  at  the  wafer  probe  as  well  as  on  the  bench.    By  and  large,  what  we  do  test  at  wafer  probe  is  a 
small  number  of  tests  compared  to  what  we  could  test,  but  the  technical  problems  and  the  measuring 
methodology,  instrumenting  systems  that  we  might  use  with  it  are  very  complex.    It  is  a  complex 
problem,  and  it  is  something  where  some  good  work  will  pay  off. 

Final  discussion  touched  on  the  dollar  impact  of  all  of  this.    There  are  apparently  many 
things  that  we  feel  that  NBS  could  do,  but  what  is  it  worth?   It  is  hard  to  get  your  hands  around  this 
to  do  a  return  on  investment  computation  for  the  money  that  might  be  spent.    Talking  in  general 
terms,  if  the  dollar  return  is  considered  in  terms  of  money  now  wasted  in  the  United  States  in  vari- 
ous ways  by  people  who  either  build  systems,  spending  a  lot  of  money  to  get  a  job  done  because  of 
their  fear  to  use  these  undefined  dissimilar  types  of  devices  that  they  can  now  buy,  or  people,  who, 
because  of  their  lack  of  understanding  for  the  specifications,  buy  the  wrong  products,  or  buy  prod- 
ucts that  do  not  work  in  their  systems,  the  combined  totals  must  be  worth  literally  tens  if  not 
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hundreds  of  millions  of  dollars.    In  one  particular  case  there  was  a  known  $200,  000  loss,  associated 
with  a  misunderstanding  of  what  the  specifications  meant. 

A.  4. 3     Sources  and  References 

(Presented  by  D.  Lawrence) 

The  Sub-Group  on  Sources  and  References  covered  five  basic  areas.    They  are:    Impulse 
testing,  pulse  and  waveform  standards,   standards  for  battery  life,  AC/DC  transfer  standards,  and 
finally  DC  standards. 

On  impulse  testing,  we  were  not  really  able  to  clearly  define  the  problem.    It  centers  on  the 
requirement  for  high  accuracy  pulses  to  determine  if  fairly  subtle  changes  have  actually  occurred. 
The  purpose  is  to  use  pulses  to  permit  analyzing  circuits  and.devices  without  actually  taking  them 
out  of  service. 

In  the  area  of  pulse  and  waveform  standards,  one  concept  is  to  provide  a  linear  ramp  gener- 
ator standard  for  working  in  the  field  of  aperture  time  measurement.    Problems  abound,  including 
trigger  jitter.    (One  participant  suggested  there  was  a  better  technique  with  which  to  accomplish  this 
measurement. )   We  make  ramps  all  the  time  with  oscilloscopes,  but  they  tend  to  fall  into  the  1  to  3% 
category,  while  the  requirements  here  probably  fall  closer  to  the  100  ppm  area  in  terms  of  the  time- 
varying  amplitude. 

Another  area  is  the  standard  reference  pulse.  This  deals  in  the  area  of  oscilloscope  testing, 
something  in  the  neighborhood  of  a  200  ps  rise  time  pulse  with  aberrations  of  less  than  1%,  with  duty 
cycle  from  10%  to  50%  and  a  repetition  rate  from  50  Hz  to  about  50  MHz  with  actual  rep  rate  uncriti- 
cal. 

Another  topic  is  the  need  for  a  generator  calibrated  in  crest  factor  rather  than  pulse  width, 
for  AC  device  evaluation. 

It  is  felt  there  is  need  to  investigate  techniques  to  determine  the  real  life  of  batteries  on  a 
predictive  basis.    Applications  areas  include  standard  cell  environmental  control  for  transportation, 
and  for  implantable  devices.    One  suggestion  was  that  some  type  of  device  be  implanted  within  the 
battery  to  check  its  chemical  condition. 

In  AC/DC  transfer  standards,  the  first  item  that  came  up  was  the  desire  to  locate  a  device  to 
replace  the  thermocouple.  Needed  are  an  increase  in  response  time,  and  wider  dynamic  range. 
Also  desired  are  an  increase  in  accuracy  at  higher  frequencies,  plus  a  means  to  reduce  DC  reversal 
error,  or  a  means  to  predict  it  so  that  it  does  not  require  constant  checking.  Further  needs  include 
reduction  in  the  uncertainty  of  AC  measurements  as  they  are  currently  set  up,  hopefully  to  get  audio 
down  to  the  area  of  less  than  10  ppm  and  a  proportionate  decrease  all  the  way  up  through  high  fre- 
quency AC  from  1  MHz  up  to  100  MHz. 

Another  point  discussed  was  the  possibility  of  considering  improving  noise  sources  as  an 
alternative  method  to  AC  calibration  over  various  frequency  bands.    The  object  would  be  to  use 
noise  as  a  method  of  getting  a  one-point  calibration  on  an  AC  measuring  device  or  AC  transfer  stan- 
dard. 

The  problem  with  standard  cells  is  that  while  they  meet  all  of  the  electrical  requirements, 
we  need  a  standard  with  more  environmental  tolerance.    Desired  range  is  from  10  to  50  C,  with  a 
spec  in  the  area  of  1  to  5  ppm.    We  would  really  like  to  see  10  volts  as  opposed  to  1  volt,  but  accept 
that  the  1  volt  is  probably  going  to  have  to  be  there  until  we  can  replace  standard  cells.    The  10-volt 
reference  is  what  everybody  really  wants,  to  get  their  signals  out  of  the  noise.    We  would  also  like 
to  see  a  wider  load  range  capability,  up  to  at  least  10  raA. 
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Transportability  and  short  warmup  time  are  needs  as  well.    The  current  one-hour  that  is 
needed  with  most  standard  references  is  probably  acceptable.    What  is  not  acceptable  is  the  1-day  to 
2 -week  settling  time  of  standard  cells. 

One  suggestion  was  for  automatic  correction  through  microprocessor  techniques,  of  the  drift 
and  temperature  coefficients.    We  are  really  looking  for  a  laboratory  quality  standard.    Many  felt 
that  a  10  ppm  standard  could  well  do  the  job  for  a  large  majority  of  firms.    In  terms  of  the  NBS  con- 
tribution, one  of  the  things,  of  course,  is  more  evaluation  of  industry  designs  along  these  lines. 
One  attendee  brought  up  band-gap  devices,  as  a  possible  substitute  for  Zeners.    In  a  way,  we  are 
looking  for  a  means  by  which  they  could  be  improved.    It  is  possibly  an  area  where  the  Institute  for 
Materials  Research  at  NBS  might  be  more  oriented.    The  big  thing  that  stands  in  the  way  of  most 
Zeners  today  appears  to  be  noise.    One  possibility  might  be  different  ways  of  doping  to  reduce  the 
noise. 

In  the  short  term,  it  might  be  useful  to  consider  a  measurement  assurance  program  for 
Zener  transfer  standards  at  the  10  ppm  level,  something  that  could  be  put  into  use  today  and  would 
meet  the  needs,  again,  of  many  users. 

A  last  point  was  that  a  good  payoff  would  result  from  investigation  of  short  term  Zener 
parameters  that  would  predict  long  term  performance.    The  current  selection  process  for  high  sta- 
bility Zeners  is  a  long,  tedious  process,  and  any  way  of  eliminating  a  lot  of  poorer  diodes  that  cur- 
rently go  into  that  testing  cycle  in  the  beginning,  would  have  a  very  definite  economic  payback. 

Q:    I  wonder  how  intense  the  feeling  was  about  a  10-volt  calibration  service?    You  mentioned 
that  it  would  be  more  useful  than  having  to  tie  back  to  1  volt.    Suppose  10  volts  were  offered  by  the 
Bureau  at  an  equivalent  accuracy  to  1  volt? 

A :    If  it  were  offered  at  equivalent  accuracy,  I  would  say  that  it  would  be  more  acceptable  to 
most  than  1  volt.    I  think  that  the  problem  of  1  ppm  being  a  microvolt  right  in  the  noise  band  is  a 
serious  difficulty.    Measurement  technique  may  be  the  hardest  thing  to  come  by,  especially  for  the 
small  firm.    Getting  signal  levels  up  is  definitely  one  way  to  make  it  easier. 

Q:    You  spoke  of  AC  to  DC  transfer.    Are  you  talking  about  voltage  or  current,  or  both,  and 
are  you  talking  about  something  that  is  faster  responding  than  thermocouples? 

A:   We  are  dealing  primarily  with  voltage  standards.    The  idea  was  to  generate  the  ability  to 
make  10  measurements  per  second  to  an  accuracy  of  0. 1%  to  100  kHz.    The  main  utility  is  in  the 
world  of  automatic  testing.    You  are  trying  to  crank  a  product  out  at  the  highest  rate.    You  are  scan- 
ning over  the  bandwidth  in  various  frequency  steps  and  the  time  that  it  takes  to  settle  for  very  high 
accuracy  is  long,  when  you  are  dealing  with  true  RMS.    It  is  either  improve  the  thermocouple,  find 
another  device,  or  something  of  that  nature. 

A.  4. 4     Systems 

(Presented  by  F.  Seeley) 

The  first  problem  talked  about  was  in  the  general  area  of  electromagnetic  interference  and 
noise.    The  kind  of  noise  discussed  in  the  main  was  noise  that  comes  in  the  power  line  as  an  unin- 
vited guest  or  self -generated  noise,  something  that  we  do  to  ourselves  from  the  computer  or  the 
peripheral,  or  whatever  other  computational  device  or  controlling  device  you  might  have  -  key- 
boards, CRT's,  other  devices,  or  the  instruments  and  power  transformers  within  the  devices  them- 
selves.   If  it  was  not  in  the  system,  you  might  not  experience  that  kind  of  noise. 

The  best  we  could  come  up  with  is  that  we  thought  it  was  someone's  job  to  attempt  to  define 
and  characterize  what  we  loosely  call  power  line  pollution.    What  are  the  types  of  things  we  see  ? 
What  is  the  recommended  practice  and  what  are  the  recommended  techniques  that  you  would  have  to 
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employ  to  either  reduce  the  problem,  or  to  get  around  the  problem  -  such  things  as  isolation, 
grounding,  shielding,  guarding,  filtering,  power  regulators,  etc.  ? 

The  other  strategy  in  solving  the  general  problem  of  noise  is  to  say,  well,  even  if  you  fol- 
lowed all  of  the  best  practices  in  the  world,  you  still  are  going  to  have  unwanted  signals.    Perhaps 
there  should  be  some  attempt  to  define  the  limits  to  which  one  could  make  precise  measurements  or 
rapid  measurements  or  system  type  measurements.    What  are  the  conditions  under  which  you  could 
make  these  measurements  in  this  noisy  environment?  What  is  the  degradation  of  the  measurement 
process  in  a  noisy  environment?  Basically,  these  are  paper  studies. 

The  other  large  class  of  problems  falls  into  the  general  area  of  how  do  you  validate  system 
performance,  really  the  performance  of  the  system  to  accomplish  its  function?   That  is  a  unique 
characteristic  of  a  system,  since  it  is  usually  designed  to  do  a  very  specific  thing  or  specific  class 
of  things,  as  opposed  to  general  purpose  instrumentation,  be  it  testing  a  blood  sample  or  testing 
missile  rounds  or  whatever. 

There  are  peculiar  things  that  exist  in  the  system  world.    There  are  the  interactions  between 
measurement  and  stimuli  within  systems  that  are  not  necessarily  present  in  looser  aggregates  of 
measurement  and  stimuli  components.    There  are  history-dependent  phenomena.    There  are  also 
time-dependent  phenomena,  and  once  in  a  while  in  systems  you  see  just  data  outliers,  glitches, 
things  that  you  don't  really  understand,  and  you  hope  will  go  away  if  you  repeat  the  test,  and  very 
often  they  do.    Those  are  some  peculiar  problems  to  get  into  when  you  are  dealing  with  systems. 

The  first  question  is  how  do  you  validate  the  performance  of  the  system  in  its  static  sense? 
In  other  words,  if  there  is  a  voltage  to  be  measured  or  a  current  to  be  measured,  or  a  resistance  to 
be  provided,  and  you  can  do  that  without  regard  to  time,  how  can  you  best  do  it?  But  you  would  like 
to  do  it  in  the  system  environment,  on  the  floor  in  the  temperature -environment  of  the  system,  with 
all  of  the  noise  present.    Some  sort  of  transportable  standards  are  needed  to  do  this,  to  bring  up  to 
the  system.    What  is  of  interest  to  people  generally  who  were  at  our  session  are  the  typical  parame- 
ters, for  example,  AC  voltages  up  to  1000  V  and  possibly  out  as  high  as  10  MHz.    There  is  also  a 
range  of  accuracy,  with  about  1%  as  the  least  accurate  type  of  standard.    We  would  like  to  have  at 
the  other  end  of  the  accuracy  spectrum  about  30  ppm,  sort  of  pushing  the  state-of-the-art. 

In  DC  voltage,  up  to  maybe  1000  V  in  a  transportable  standard  are  needed  for  static  testing, 
somewhere  in  the  uncertainty  ranges  of  0. 1%  to  on  the  order  of  10  ppm. 

Most  systems  people  talk  about  measuring  resistances  and  providing  stimuli  on  the  order  of 
milliohms  to  about  100  M,  and  these  range  generally  from  1%  to  as  good  as  10  ppm.    Current  and 
power  are  also  of  interest. 

Most  people  felt  that  pulse  characterization  or  pulse  characteristics  are  necessary  in  some 
sort  of  transportable  standard.    Either  as  measured  or  stimulated  system  parameters,  people  seem 
to  be  interested  in  rise  time,  fall  time,  decay  time,  flatness  of  the  amplitude,  flatness  across  the 
duration  of  the  pulse. 

Very  often  one  will  measure  a  quantity  within  a  circuit  -  say  at  the  3 00- volt  level,  and  then 
within  a  few  milliseconds  attempt  to  measure  a  5  V  or  lower  level  in  another  circuit.    There  may 
well  be  stored  energy  in  that  circuit,  and  you  can  either  cause  a  tremendous  error  in  performing  the 
next  test,  or  you  might  have  so  much  stored  energy  that  you  could  cause  the  device  you  are  trying  to 
test  to  actually  fail.    Results  can  be  either  absolute  failure  via  the  system  pulse  inducing  a  failure 
mode  in  the  device  under  test,  or  a  pseudo -failure,  where  you  think  it  is  bad,  and  it  really  is  good. 

Those  are  the  sorts  of  things  in  which  you  get  involved,  where  things  are  no  longer  time- 
independent,  and  become  very  definitely  time -dependent. 
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What  people  in  our  session  felt  were  needed  then  were  things  that  you  might  call  dynamic 
standards.    What  might  be  needed  is  some  technique  for  measuring  the  time -dependence  of  program- 
mable DC  sources  and  AC  sources.    Basically  this  is  the  problem  of  settling  versus  steady  state. 
How  long  does  it  take  to  get  a  steady  state  value  out  of  these  sources,  or  when  you  try  to  catch  them 
on  the  fly,  what  sort  of  uncertainties  does  one  run  into? 

The  next  thing  covered  was  how  does  one  characterize  the  sources  used  within  systems  or  the 
measuring  devices  used  within  system  interfaces? 

We  also  wanted  to  suggest  that  what  else  was  needed  was  the  development  of  techniques  for 
measuring  pulse  characteristics  in  digital  logic  testers.    Here  is  a  very  knotty  technical  problem 
where  you  have  many  digital  words  coming  in  at  a  4  megaHerz  rate,  and  you  would  like  to  basically 
characterize  the  analog,  the  amplitude,  the  rise  time,  the  fall  time,  the  pulse  duration  of  each  one 
of  these  pulses  in  this  dynamic  environment  on  these  many  lines.    As  you  see,  we  have  rise,  fall, 
amplitude,  skew  and  other  characteristics  of  that  nature. 

Q:   Did  you  cover  anything  on  the  interface  interconnection  between  devices  under  test  and 
the  system  itself,  the  connection  problems,  and  so  on,  internal  system  cabling? 

A:   We  did  talk  about  that.    We  were  trying  to  characterize  the  system's  performance  either 
in  the  static  sense  or  the  dynamic  sense,  and  it  is  implied  that  it  is  at  the  system  interface. 

Q:   The  problem,  if  you  will,  that  exists  quite  frequently  is  that  a  device  plugged  into  a  sys- 
tem is  not  the  same  as  a  device  connected  in  its  actual  application  mode.     For  instance,  quite  fre- 
quently in  a  system  in  the  digital  world,  you  have  got  to  go  a  little  bit  farther  between  a  board  and  a 
test  device,  because  you  are  doing  different  things  to  evaluate  those  lines.    The  loading  is  different. 
So  the  results  of  the  pulses  that  are  generated  are  different. 

Q:   There  was  also  talk  about  an  advanced  portable  standard,  both  AC  and  DC,  with  different 
dynamic  loads,  since  the  loading  would  change. 

A:   That  sounds  like  an  example  of  going  through  and  making  measurements,  and  then  actu- 
ally inducing  an  erroneous  result  in  the  circuit  under  test,  because  of  energy  transferred  from  your 
system.    That  could  just  as  well  be  a  loading  problem. 

Q:    I  wondered  what,  if  any,  conclusions  were  to  come  on  the  subject  of  software  and  whether 
anybody  here  has  any  comments  on  software? 

A:   That  was  mentioned,  but  probably  implicit  in  the  lack  of  discussion  on  it  was  our  overrid- 
ing feeling  that  software  becomes  really  a  part,  and  almost  the  sole  province  of  the  company  devel- 
oping the  product.    I  am  speaking  personally  though.    That  did  not  come  out  in  our  discussion. 

Q:   Well,  I  think  both  in  terms  of  hardware  interfacing  and  software,  most  people  thought  it 
would  be  the  problem  of  the  person  doing  it.    There  were  not  enough  generalities  that  you  could  ever 
do  anything  actually  to  help  the  problem  out.    You  might  want  to  define  some  tasks,  I  think,  in  terms 
of  operating  systems,  but  not  down  at  the  programming  level. 

Q:    I  would  like  to  emphasize  that  one  of  the  things  that  was  discussed  was  the  need  for  the 
ability  to  calibrate  in  place  and  essentially  without  disturbing  the  system.    When  you  realize  that  you 
have  a  minimum  of  $100,  000,  perhaps  three  or  four  hundred  thousand  dollars  tied  up  in  a  complex 
test  system,  you  simply  do  not  want  to  take  it  down  and  remove  an  instrument  to  calibrate  it. 

Secondly,  it  may  be,  as  has  been  mentioned,  because  of  the  stray  parameters  that  exist  in 
your  connection  system,  that  the  performance  you  are  getting  at  the  unit-under-test  terminals  is  not 
the  same  as  the  instrument  calibration. 
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Related  to  this  is  our  self -test  or  self -calibration  program.    The  process  of  calibrating  in 
place  is  best  done,  most  conveniently  done,  by  a  program  on  the  system  itself.    Developing  a  tech- 
nique which  is  truly  self -calibrating,  using  one  transportable  standard  or  a  number  of  transportable 
standards,  is  something  that  perhaps  the  Bureau  could  assist  in,  at  least  in  describing  algorithms, 
which  would  indeed  provide  you  the  calibration  that  is  traceable  to  the  Bureau  of  Standards. 

A:    In  the  area  of  algorithms  and  processes  and  technique  control,  the  Bureau  has  always  his- 
torically published  how  it  gets  from  basic  units  of  measurement,  the  few  SI  units  out  to  all  the  other 
parameters.    It  publishes  the  work  and  how  you  scale  up  and  down.    But  when  you  get  to  the  auto- 
matic area,  if  you  do  that  in  the  system,  immediately  there  is  a  new  technology  involved.    I  am  not 
sure  whether  that  is  really  a  systems  problem  or  a  components  problem,  but  it  is  a  problem  and  it 
needs  to  be  looked  at.    Some  people  have  looked  at  it  and  done  some  things.    Others  have  not.    To 
act  as  a  clearing  house  would  be  a  very  useful  function. 

Q:  It  almost  sounds  as  if  in  a  way  you  have  laid  the  groundwork  for  the  problems  in  systems 
which  might  be  worth  the  Bureau's  while  to  look  into. 

A:    I  would  agree.    When  you  start  seeing  collections  of  the  same  sorts  of  problems  in  differ- 
ent systems,  then  you  can  quantify  it  further  than  we  have  attempted  to  do  here.    So  it  is  a  frame- 
work. 

Q:  Did  anyone  bring  up  the  possibility  of  the  Bureau  sending  around  some  kind  of  generalized 
audit  package  that  might  be  used  in  validating  certain  classes  of  systems? 

A:    Yes.    If  we  had  transportable  standards,  it  would  help.    We  covered  them,  and  their 
parameters.    But  the  process  of  tying  them  to  the  system  is  the  problem.    These  are  the  parame- 
ters, but  what  is  the  interface  to  the  system?   It  depends  on  the  systems. 

One  successful  example  is  one  company's  automatic  network  analyzer,  which  was  probably 
the  only  automatic  network  analyzer  for  making  measurements  from  100  MHz  to  12.4  GHz.    There 
is  a  community  of  users  who  pass  standards  around,  but  the  problem  is  well-defined.    It  is  strictly 
one  machine  with  one  set  of  software.    In  the  more  general  case  you  have  a  lot  of  different  types  of 
machines  and  a  lot  of  different  applications. 

A.  5     Summary  of  Post-Workshop  Written  Feedback  Reports 

At  the  immediate  conclusion  of  the  Workshop,  the  twenty-four  attendees  were  asked  to  fill  out 
forms  briefly  summarizing,  in  priority  order,  the  two  or  three  most  critical  electrical  measure- 
ment problems  and/or  problem  areas  which  they  believed  the  Electricity  Division  should  pursue. 
Nineteen  responses  were  received.    Since  there  were  almost  seventy  separate  items  listed,  only  a 
few  received  more  than  several  "votes".    This  was  more-or-less  to  be  expected  in  view  of  the 
diversity  of  the  group  that  was  invited. 

Since  categories  were  not  suggested,  the  way  in  which  items  were  expressed  differed  some- 
what from  writer  to  writer.  The  following  summary  ignores  priorities  unless  they  seemed  particu- 
larly significant. 

1.  Investigate  and/or  develop  transportable  standards  suitable  for  validating  the  dynamic 
performance  of  automated  systems.    (10  entries,  8  in  first  priority. ) 

2.  Same  as  (1),  omitting  dynamic  requirement.    (3  entries,  all  in  first  priority. ) 

3.  Define  parameters  for  system  building  blocks  and  A/D  interfaces  (such  as,  but  not  lim- 
ited to,  ADC's  and  DAC's),  to  provide  basis  for  developing  test  and/or  calibration  tech- 
niques, including  dynamic  performance.    (8  entries,  3  first,  3  second  priorities. ) 
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4.  Set  up  definitions  for  dynamic  or  time  domain  measurement  parameters,  such  as  settling 
time,  acquisition  and  aperture  times,  fully  specifying  the  quantities.    (6  entries.) 

5.  Carry  out  long-term  stability  studies  for  components.    (6  entries.) 

6.  Develop  measuring  techniques  for  noise  measurements  -  standards  and  characterization. 
(5  entries.) 

7.  Develop  measuring  techniques  for  pulse  response  testing,  including  a  "pure"  pulse  gen- 
erator, step  or  ramp  generators,  even  exponential  pulses.    (4  entries. ) 

8.  Develop  measuring  techniques  for  settling  time.    (3  entries.) 

9.  Investigate  and/ or  develop  solid  state  DC  voltage  standards  at  1  to  10  volts,  with  2-5  ppm 
accuracy.    (3  entries. ) 

10.  Develop  measuring  techniques  for  pulsed/ settling  time  measurements  for  resistors.    (2 
entries. ) 

11.  Investigate  the  possibility  of  improving  the  stability  of  film  resistors.    (2  entries. ) 

12.  High-resistance  calibration  -  improved  standards  and  methodologies  are  needed.    (2 
entries. ) 

Remaining  entries  appeared  but  once.    In  no  special  order,  they  were: 

13.  Develop  computer-driven,  dynamic  source  to  exercise  automatic  measuring  system  for 
DC  volts,  current,  resistance. 

14.  Develop  measuring  techniques  for  pulsed- settling  time  measurements  on  capacitors. 

15.  Set  up  MAP  for  transportable  DC  voltage  standards,  at  2  to  5  ppm  accuracy  level. 

16.  Reduce  AC  measurement  uncertainties. 

17.  A  broader  range  of  AC  standards  is  required. 

18.  An  alternative  to  the  standard  cell,  with  standard  cell  long  term  stability  but  none  of  its 
other  problems,  is  required. 

19.  NBS  should  carry  out  work  on  dielectric  absorption. 

20.  Power  line  characterization  is  of  great  importance. 

21.  Work  on  lifetimes  for  power  sources  for  medical  implants  would  be  very  useful. 

22.  A  better  power  measuring  capability  is  required. 

23.  A  phase  measuring  capability  is  also  needed. 

24.  NBS  should  extend  MAP's  in  order  to  improve  and  clarify  traceability. 

25.  NBS  should  provide  leadership  in  publishing  suggested  procedures  for  testing  automated 
systems. 
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26.  Higher  frequency  standards  of  C,  R,  are  required. 

27.  Perform  research  on  the  long-term  stability  of  bipolar  transistors  -  V      versus  time, 
in  particular. 

28.  Carry  out  consumer  product  testing,  e.g.,  hearing  aids  for  frequency  response,  distor- 
tion, noise. 

29.  Medical  equipment  calibrations  for  hospitals  would  be  valuable. 

30.  Temperature  coefficient  standards  down  to  0. 1  ppm/   C  are  needed. 

31.  Develop  binary  voltage  divider  calibrations  to  1  ppm  accuracy  for  ratio. 

A.  6     Summary  of  Post-Workshop  Written  Commentaries 

At  the  immediate  conclusion  of  the  Workshop,  the  attendees  were  also  asked  to  fill  out  forms 
briefly  summarizing  their  reactions  to  the  Workshop  itself,  and  to  indicate  their  suggestions  for 
future  activities.    What  follows  is  a  summary  of  the  eighteen  inputs  received. 

1.  Ten  participants  indicated  that  they  felt  the  Workshop  met  its  intended  goals  without 
reservation,  six  indicated  that  they  felt  it  at  least  partially  met  them,  while  one  indi- 
cated that  he  felt  it  had  not  met  its  goals. 

Two  comments  indicated  that  the  scope  of  the  Workshop  might  have  been  a  little  too 
broad. 

2.  Thirteen  participants,  the  total  number  responding  to  a  question  regarding  the  worth  of 
the  Workshop  for  the  attendees  themselves,  indicated  that  it  was  indeed  worthwhile. 
Three  of  the  responses  were  qualified  as  conditional  upon  the  NBS  implementation  of 
Workshop  recommendations. 

Comments  included  the  fact  that  the  Workshop  helped  delineate  industry  problems,  and 
helped  industry  more  fully  understand  the  work  of  the  Bureau. 

3.  Answers  to  a  question  relating  to  ways  in  which  the  Workshop  could  have  been  improved 
covered  a  range  of  comment.    There  was  no  single  most  popular  critique,  and  the  inputs 
ran  the  gamut  that  might  be  expected  from  so  diverse  a  group.    Many  should  prove  help- 
ful in  organizing  future  efforts  of  this  nature.    They  include: 

Include  more  device  users. 

NBS  should  take  a  more  positive  stand  in  measurement  assurance  -  "traceability". 

It  would  have  been  helpful  to  hold  the  tour  and  NBS  activity  descriptions  before  the  Work- 
shop. 

More  could  perhaps  have  been  gained  via  prior  preparations  by  the  invitees.    (2  com- 
ments to  this  effect. ) 

Perhaps  area-limiting  guides  are  needed  to  narrow  the  discussions. 

Have  wider  representation. 

Provide  better  understanding  of  what  NBS  can  do. 
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Define  NBS  role  better  vis-a-vis  industry.    (2  comments. ) 
Narrow  down  subject  matter,  perhaps  for  in-depth  sessions. 

4.  Twelve  attendees  indicated  that  what  they  liked  most  about  the  Workshop  was  the  profes- 
sional interaction,  the  stimulation,  the  openness  of  idea  exchange  and  the  general  infor- 
mality of  the  technical  sessions. 

Three  participants  most  appreciated  information  regarding  Bureau  capabilities,  goals 
and  plans. 

One  respondent  was  most  favorably  impressed  by  the  Workshop  organization,  and  its 
informative  nature. 

5.  Adverse  comments  were  relatively  few: 
Too  many  sub-groups. 

Tour  too  short  and  not  sufficiently  relevant. 

Lack  of  clear  understanding  of  Workshop  goals. 

Some  emphasis  on  products  rather  than  ideas. 

Insufficient  time. 

Not  all  companies  sent  their  most  knowledgeable  people. 

Need  for  more  advance  preparation  of  attendees,  particularly  sub-group  chairmen. 

6.  Twelve  attendees  favored  a  followup  workshop  in  a  few  years  or  less,  with  several  indi- 
cating a  desire  for  intervals  as  short  as  a  year  or  even  less,  possibly  for  working-group 
meetings. 

One  specific  suggestion  was  for  interdisciplinary  workshops,  including  two  or  more 
cooperating  NBS  Divisions. 

Another  comment  suggested  followup  reports  to  the  participants  on  Division  progress  in 
the  areas  identified. 

7.  Suggestions  for  subject  matter  and  format  for  new  workshops  included  six  requests  for 
more-or-less  identical  workshops.    Two  requested  that  new  workshops  include  a  review 
of  NBS  activities  since  the  last  one. 

Additional  suggestions  were  forthcoming  for  restructuring  and  reformatting  new  work- 
shops, as  follows: 

Have  greater  systems  orientation,  including  software. 

Do  not  restrict  to  predetermined  sub-groups. 

Include  greater  description  of  NBS  existing  and  planned  programs. 

Suggest  formal  presentations  by  invitees  themselves. 
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Have  more  clearly  defined  objectives. 

Include  other  industry  areas. 

Plan  for  different  working  groups,  including  semiconductors. 
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Appendix  B :     Mi scellaneous 

B.  1     Final  Workshop  Agenda 

23  September  1974  (Monday) 

9:00  to    9:05  AM  Welcome,  Dr.  Arthur  O.  McCoubrey,  Director,  Institute  for  Basic 

Standards. 

9:05  to    9:40  AM  Introduction  to  NBS  and  the  Electricity  Division,  Dr.  Barry  N.  Taylor, 

Chief,  Electricity  Division. 

9:40  to  10:00  AM  Workshop  scope,  goals,  and  modus  operandi,  Peter  Richman,  Consult- 

ing EE. 

10:00  to  10:30  AM  Coffee  break  and  informal  discussions. 

10:30  to  12:00  PM  Open  discussion  based  on  material  submitted  by  attendees  prior  to 

Workshop  (led  by  P.  Richman).    Initial  working  sub-groups  finalized. 

12:00  to    1:00  PM  Lunch 

1:00  to    3:30  PM  Begin  detailed  discissions  of  specific  topics  in  sub-groups. 

3:30  to    4:00  PM  Coffee  and  informal  discussions.    Attendees  to  have  opportunity  to 

indicate  their  Tuesday  sub-group  preference  and  to  suggest  possible 
new  sub-groups. 

4:00  to    5:30  PM  Tour  of  the  Electricity  Division. 

7:00  to       ?  Dinner  and  informal  discussions. 


24  September  1974  (Tuesday) 

9:00  to    9:30  AM  Brief  review  of  Monday's  sessions,  consideration  of  possible  new  sub- 

groups, and  finalization  of  Tuesday's  sub-group  membership. 

9:30  to  12:00  PM  Sub-groups  continue  to  meet;  attempt  to  reach  consensus  and  to  delin- 

(Coffee  available)  eate  critical  needs  and  problems  in  detail. 

12:00  to    1:30  PM  Lunch,  informal  discussions,  and  individual  preparation  for  feedback 

session. 

1:30  to    3:30  PM  Feedback  session.    Sub-group  chairmen  report  back  to  assembled  par- 

ticipants, with  open  discussion  (led  by  P.  Richman). 

3:30  PM  Formal  adjournment.    The  remainder  of  the  day  will  be  available  for 

additional  discussions,  tours,  etc. ,  at  the  option  of  the  attendees. 
(We  cordially  invite  any  attendee  who  so  desires  to  stay  an  extra  day 
and  visit  the  Division  on  Wednesday,  25  September. ) 
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B.  2     Tour 

A  tour  of  some  of  the  Electricity  Division's  projects  was  arranged  as  part  of  the  Workshop. 
The  following  brief  descriptions  of  each  of  the  activities  visited  were  distributed  to  all  of  the  atten- 
dees prior  to  the  tour. 

Josephson  Volt  Maintenance  and  Cryogenic  Comparators 

Since  1972,  the  U.  S.  legal  volt  has  been  defined  and  maintained  at  NBS  to  an  accuracy  of  a  few 
parts  in  10  via  the  AC  Josephson  effect,  an  effect  which  involves  the  quantum -mechanical  tunneling 
of  electron  pairs  in  superconductors.  In  the  voltage  standard  application,  a  Josephson  tunneling 
device  is  used  as  a  precise  frequency-to-voltage  transducer  with  a  proportionality  constant  equal  to 
2e/h.  Presently  under  development  is  an  all-cryogenic  voltage  standard  system  featuring  a  reliable 
thin-film  Josephson  device,  a  cryogenic  current  comparator,  a  cryogenic  resistive  divider,  a  SQUID 
null  detector  and  superconducting  switches. 

Solid  State  References  and  Automated  Voltage  Measurements 

Present  studies  of  commercial  reference  units  are  being  carried  out  with  the  objectives  of:   (a) 
Determining  the  feasibility  of  using  them  to  disseminate  the  volt  at  the  3  to  10  ppm  level;  (b)    Obtain- 
ing quantitative  performance  data  on  the  devices;  and  (c)   Providing  data  for  further  improving  the 
performance  of  solid  state  references.    Computer  automation  of  low  level  voltage  measurements  will 
improve  NBS  capability  for  maintaining  and  disseminating  the  volt.    The  present  effort  is  being 
directed  toward  developing  an  automated  measuring  system  utilizing  the  HP9830  calculator,  the 
MIDAS  controller  and  the  NBS  developed  low-level  switch. 

High  Voltage  Pulses  and  Kerr  Effect 

The  electro-optic  Kerr  effect  is  being  used  for  the  measurement  of  high  voltage  pulses. 
Development  is  currently  concentrated  in  three  areas:   (a)   Accurate  calibration  of  Kerr  systems; 
(b)   Investigation  of  the  properties  of  liquids  used  in  Kerr  systems;  and  (c)    Computer  automation  of 
high  voltage  pulse  measurements.    The  objective  of  this  work  is  to  develop  and  disseminate  methods 
of  measuring  high  voltage  pulses  -  pulses  with  rise-times  of  1  ms  to  1  ns  and  peak  values  from  one 
thousand  volts  to  several  million  volts  -  with  an  uncertainty  of  less  than  1%  of  their  peak  value. 

Direct  Reading  AC  (RMS)  Voltage  Standard 

A  standard  for  measuring  RMS  voltage  in  the  range  5  V  to  1000  V  from  50  Hz  to  20  kHz  is 
presently  under  development  with  extension  of  the  ranges  to  higher  and  lower  values  to  follow.    An 
automated  AC-DC  transfer  circuit  compares  the  AC  voltage  with  a  DC  reference  and  hence  to  the 
legal  volt.    The  system  will  make  possible  multi-point  calibration  of  precision  AC  sources  to  an 
expected  accuracy  of  20  ppm  and  stability  measurements  to  5  ppm. 

Sampling  Wattmeter 

A  digital  and  computerized  system  was  recently  completed  for  the  measurement  of  electrical 
power  with  an  uncertainty  no  greater  than  0.  02%  up  to  2  kHz.    Instantaneous  samples  of  voltage  and 
current  are  collected  in  pairs  at  pre-determined  intervals  and  the  corresponding  numerical  values 
stored  in  a  computer  in  real  time.    Average  power  is  calculated  by  numerical  integration  of 
current-voltage  products.    Adaptation  of  the  system  to  the  solution  of  problems  associated  with  non- 
sinusoidal  waves  is  planned,  e.g.,  those  encountered  in  distribution  and  control  systems  where 
SCR's  are  extensively  used. 
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Automated  Resistance  Measurements 

Computer  automation  of  the  process  for  calibrating  standard  resistors  is  proceeding  for  two 
separate  ranges  of  resistance  values.     From  0. 1  to  10  ohms,  a  digital  nanovoltmeter  in  conjunction 
with  a  current -comparator  potentiometer  will  be  used  to  automatically  compare  voltage  drops  across 
resistors  in  a  series  chain.    Between  100  and  10    ohms,  resistors  will  be  intercompared  using  an 
unbalanced,  4-terminal  bridge  technique.    A  digital  voltmeter  will  measure  both  the  unbalanced  volt- 
age and  the  bridge  current.    In  both  cases,  switching  will  be  done  with  numerically  controlled  cross- 
bar switches  using  four  of  the  six  circuits  in  the  crossbar  as  an  active  guard  to  reduce  leakage. 

B.3     Working  Sub-Groups 

The  following  table  gives  the  (approximate)  sub-group  participation  during  the  course  of  the 
Workshop.    (But  some  Electricity  Division  staff  other  than  those  shown  attended  the  sub-group  ses- 
sions as  observers  on  a  more  or  less  random  basis. )   Session  1  was  that  held  on  Monday,  24  Sep- 
tember, 1:00  to  3:30  PM;  and  sessions  2A  and  2B  were  those  held  on  Tuesday,  25  September,  from 
approximately  9:30  to  10:30  AM  and  10:45  to  12:00  PM,  respectively.    The  four  sub-groups  were 
numbered  as  follows,  and  these  numbers  are  used  in  the  table. 

I.  Components  and  Parameter  Measurements 

II.  Signal  Conditioning  and  Data  Conversion 

III.  Sources  and  References 

IV.  Systems 


Attendee' s 

Session 

Attendee's 

Session 

Name 

Affiliation 

1       2A      2B 

Name 

AffiUation 

1        2A     2B 

L.  Baltz 

Ford 

IV      II       III 

D.  Schneider 

Fluke 

IV      IV      IV 

E.  Boeckmann 

Angstrohm 

Chmn.Sub-Gr.  I 

F.  Seeley 

U.S.  Army    Chmn.Sub-Gr.  IV 

W.  ColUer 

Tektronix 

IE      III      II 

R.  Shaw 

Westinghouse 

I         II        IV 

T.  Coughlin 

G-R 

I         IV      II 

L.  Smith 

Analog  Dev. 

II        III      III 

J.  Dougherty 

USAF 

I         III      I 

D.  Strain 

ESI 

I         II        II 

G.  Herron 

Keithley 

II       IV      IV 

R.  Talambiras 

Analogic 

II        IV      IV 

D.   Lawrence 

H-P 

Chmn.Sub-Gr.  Ill 

D.  Terrett 

ADT  Sec.  Sys. 

IV      II        I 

D.  Ludwig 

Teledyne  P 

Chmn.Sub-Gr.  II 

J.  Tripp 

National  Semi. 

II        II        III 

E.  Mueller 

FDA 

III      I         I 

R.  Verity 

L&N 

I         III      II 

W.  Plice 

Honeywell 

in     ii      ii 

W.  Walters 

Codi 

m      I        I 

R.  Raybold 

NBS 

IV      II        IV 

M.  Young 

Teradyne 

II        IV      IV 

B.  Renz 

AEPSC 

IV      II        II 

R.   Zuck 

Geometric  Data  IV 

Electricity  Divi 

sion  Participants 

M.  Oldham 

NBS  Recorder, 

Sub-Gr.  I 

W.  Eicke 

NBS  Participant,  Sub-Gr.  I 

T.  Souders 

NBS  Recorder, 

Sub-Gr.  II 

H.  Schoenwetter    NBS  Participant,  Sub-Gr.  II 

B.  Field 

NBS  Recorder, 

Sub-Gr.  Ill 

R.  Turgel 

NBS  Participant,  Sub-Gr.  Ill 

R.  Kleimann 

NBS  Recorder, 

Sub-Gr.  IV 

J.  Morrow 

NBS  Participant,  Sub-Gr.  IV 
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Recognizing  the  proliferation  of  sophisticated  modern  electronic  instrumentation  in  the 
field  of  electrical  measurements,    the  Electricity  Division  of  the  National  Bureau  of 
Standards  recently  initiated  a  new  program  in  the  general  area  of  dynamic  measure- 
ments and  standards   in  support  of  such  instrumentation.      Recognizing  further  that  the 
vastness  and  complexity  of  the  field  -would  require,    at  the  earliest  stages  of  the  progr 
identification  of  the  most  critical  problem  areas,    the  Electricity  Division  held  a  work- 
shop on  23  and  24  September,    1974,    at  the  Bureau's  Gaithersburg  site,    to  assist  it  in 
ascertaining  just  what  these  areas   in  fact  were.      The  basic  idea  of  the  Workshop  was 
to  bring  together  a  broadly  representative  group  of  some  twenty-five  leading  manufact- 
urers  and  prime  users,    working  in  a  free  and  open  atmosphere,    in  order  to  have  them 
delineate  the  present  and  future  critical  support  needs   in  the  field  of  dynamic  electrica 
measurements  for  modern  electronic  instrumentation,    with  emphasis   on  physical 
standards,    standardized  measurement  methods,    new  calibration  and  measurement 
assurance  services,    relevant  data,    and  most  important,    new  measurement  methodol- 
ogies.     The  overall  objectives  of  the  Workshop  were   generally  met,    and  a  number  of 
significant  specific  programs  and  projects  consistent  with  the  mission  of  the  Electrici 
Division  were  identified.      Three  categories  broadly  cover  the  needs  as  defined  at  the 
Workshop: 
1.      The  need  to  introduce  time  as  a  measurement  parameter  has  resulted  from  the 
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requirements  of  automatic  test  and  control  systems.      Specific  areas 
considered  critical  are: 

a.  Pulsed  component  measurements. 

b.  Dynamic  performance  characterization  for  modern  signal  conditioning 
and  data  conversion  devices:     Digital-to- analog  and  analog-to-digital 
converters,    sample- and- hold  amplifiers ,    comparators,    etc.      Required 
measurements  include  settling,    aperture  and  acquisition  times.      Basic 
new  capabilities  will  be  needed,    including  precision,    non- sinusoidal 
waveform  generation  and  "standard"  digital-to-analog  converters. 

c.  Methodologies  and  techniques  for  characterizing  precision  AC  and  DC 
sources  and  measurement  devices  with  respect  to  settling  time. 

The  emergence  of  measurements  into  the   "real    world"  or  the  production 
line  via  the  automatic   system  has   introduced  a  host  of  new  parameters. 
Areas   requiring  significant  effort  include: 

a.  Investigation  of  transportable  standards  for  validation  of  static  and 
dynamic  system  performance:    AC  and  DC  voltage,    impedance,    pulses, 
settling  times. 

b.  Techniques  for  characterizing  sources  and  measurement  devices  with 
respect  to  switched  or  dynamic  loading. 

c.  Prediction  of  long-term  performance,    reliability  and- lifetime  from 
short-term  evaluation  for  a  host  of  passive  components,    semiconductors, 
and  signal  sources. 

d.  Noise  standards  and  methodologies. 

e.  Continuing  effort  to  improve  transportable   DC  standards,    and  dissemination 
at  higher  voltage  levels. 

f.  Characterization  of  the  AC  line  voltage-waveforms,    under-    and  over- 
voltages,    transients,    dynamic   impedance,    etc. 

g.  Inclusion  of  a  capability  for  environmental  control  and  variation  for  devices 
and  instruments  under  calibration,    in  all  new  work  undertaken  by  the  Bureau. 

h.      Extension  of  Measurement  Assurance   Programs  (MAP's),    especially  into 
the  above  new  areas. 

5.      Recent  work  at  the  leading  edge  of  measurement  technology  in  electronics  has 
resulted  in  the  need  for  new,    or  higher  accuracy  measurements.      For  example: 

a.  Increased  dependence  on  capacitors  as   storage  devices  has  made  dielectric 
hysteresis  a  parameter  of  importance.      Significant  measurement  work 
should  be  started  in  this  area. 

b.  Phase  difference  measurements  to  extremely  high  accuracies  are  needed  to 
calibrate   industrial  instrumentation. 

c.  Higher  accuracy  power  measurements,    particularly  for  non-sinusoidal 
waveforms,    are  needed  to  calibrate  instrumentation  used  in  the  field. 

d.  Non- sinusoidal,    high  crest-factor  precision  signals  are   needed  to  calibrate 
true  RMS  converters  and  meters. 

e.  High  accuracy  electronic  instrumentation  requires   reduction  in  AC  cali- 
bration uncertainties  to  10  ppm,    at  least  over  the  audio  range. 


Longer-term,    "frontier"  areas  in  each  of  the  above  groups  were  also 
identified.      They  are  not  specified  in  the  above  listings   since  they  involve  work 
at  still  more  basic  levels,    on  ground  that  is  presently  terra  incognita.     Never- 
theless,   they  furnish  insight  into  the  overall  technical/philosophic  climate,    and 
are  useful  for  that  reason.      Items  in  this  category  include: 

1.  Measurement  problems  in  systems,    including  effects  of  history,    measurement 

interactions,    self- calibration,    etc. 

2.  Evaluation  and  standardization  of  system  software. 

3.  Calibration  of  medical  equipment. 

4.  Replacement  of  standard  cells. 

5.  Replacement  of  thermal  AC/DC  converters. 

6.  Non-swept  but  broad- response  measurements- -  impulse,    noise,    other. 

7.  Basic  work  on  the  physics  of  dielectric  absorption;     component  drift; 
device  noise;     failure  mechanisms;     switches  and  relays. 

8.  Provision  of  improved  measurement  accuracy  for  virtually  all  dynamic 
measurements,    at  the  wafer  probe  level. 

In  addition  to  the  identification  of  these  specific  and  general  needs,   the  Work- 
shop provided  strong  evidence  that  both  the  manufacturing  and  user  communities 
in  the  electronic  instrumentation  field  look  to  the  Electricity  Division  to  provide 
the  technological  leadership  in  these  newer  areas  that  it  has  previously  furnished 
in  the  more  traditional  electrical  measurement  areas.      Indeed,    there  was  a  good 
deal  of  support  for  the  idea  of  keeping  the  electronics  community  informed  of 
Division  work-in-progress,    with  emphasis  on  projects  targeted  by  the  Workshop. 
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